Answer: \(n=12,\ a=4,\ b=64\).
We start with the main method. Use the binomial theorem to identify only the terms needed, rather than expanding the whole expression.
(a) The first term in
\(\left(a+\frac xa\right)^n\)
is \(a^n\). Since the expansion begins with \(b^4\),
\(a^n=b^4.\)
The coefficient of \(x\) is
\(\binom n1 a^{n-1}\frac1a=na^{n-2}.\)
This is equal to \(48b^3\), so
\(na^{n-2}=48b^3.\)
From \(a^n=b^4\), we have \(b^3=a^{3n/4}\). Therefore
\(na^{n-2}=48a^{3n/4}.\)
Divide by \(na^{3n/4}\):
\(a^{n/4-2}=\frac{48}{n}.\)
Squaring both sides gives
\(a^{n/2-4}=\left(\frac{48}{n}\right)^2.\)
(b) The third term is
\(\binom n2a^{n-2}\left(\frac xa\right)^2 =\frac{n(n-1)}2a^{n-4}x^2.\)
It is given as \(1056b^2x^2\), so
\(\frac{n(n-1)}2a^{n-4}=1056b^2.\)
Since \(b^2=a^{n/2}\),
\(\frac{n(n-1)}2a^{n-4}=1056a^{n/2}.\)
Thus
\(a^{n/2-4}=\frac{2112}{n(n-1)}.\)
Using the result from part (a),
\(\left(\frac{48}{n}\right)^2=\frac{2112}{n(n-1)}.\)
So
\(\frac{2304}{n^2}=\frac{2112}{n(n-1)}.\)
Hence
\(2304(n-1)=2112n,\)
which gives
\(n=12.\)
Now
\(a^{12/2-4}=a^2=\left(\frac{48}{12}\right)^2=16,\)
so \(a=4\).
Finally, \(a^n=b^4\), so
\(4^{12}=b^4.\)
Therefore
\(b=64.\)
This completes the solution and gives the required result.