Answer: \(a=3\), \(b=\pm\sqrt{6720}=\pm8\sqrt{105}\).

The expansion is

\(5(2+ax)^n.\)

The constant term is

\(5\cdot2^n.\)

This is given as \(640\), so

\(5\cdot2^n=640.\)

Thus

\(2^n=128=2^7,\)

so

\(n=7.\)

Now consider the coefficient of \(x^2\). In \((2+ax)^7\), the \(x^2\) term is

\(\binom72 2^5(ax)^2.\)

After multiplying by \(5\), the coefficient of \(x^2\) is

\(5\binom72 2^5a^2.\)

Since this is \(30240\),

\(5\cdot21\cdot32\cdot a^2=30240.\)

So

\(3360a^2=30240,\)

and hence

\(a^2=9.\)

Therefore \(a=3\) or \(a=-3\).

The coefficient of \(x\) is

\(5\binom71 2^6a=5\cdot7\cdot64a=2240a.\)

This coefficient is given as \(b^2\), which must be non-negative. If \(a=-3\), the coefficient \(2240a\) would be negative, impossible for \(b^2\). Therefore

\(a=3.\)

Then

\(b^2=2240(3)=6720.\)

Hence

\(b=\pm\sqrt{6720}=\pm8\sqrt{105}.\)