Answer: (a) \(\cos 5\theta=16\cos^5\theta-20\cos^3\theta+5\cos\theta\).

(b) The roots are

\(x=\cos\left(\frac{\pi}{20}\right),\;\cos\left(\frac{9\pi}{20}\right),\;\cos\left(\frac{17\pi}{20}\right),\;\cos\left(\frac{25\pi}{20}\right),\;\cos\left(\frac{33\pi}{20}\right).\)

(a) By de Moivre's theorem,

\((\cos\theta+i\sin\theta)^5=\cos5\theta+i\sin5\theta\).

Taking real parts,

\(\cos5\theta=\Re\big((\cos\theta+i\sin\theta)^5\big).\)

Expanding,

\((\cos\theta+i\sin\theta)^5=\cos^5\theta+5i\cos^4\theta\sin\theta-10\cos^3\theta\sin^2\theta-10i\cos^2\theta\sin^3\theta+5\cos\theta\sin^4\theta+i\sin^5\theta\).

So

\(\cos5\theta=\cos^5\theta-10\cos^3\theta\sin^2\theta+5\cos\theta\sin^4\theta\).

Now use \(\sin^2\theta=1-\cos^2\theta\), so \(\sin^4\theta=(1-\cos^2\theta)^2=1-2\cos^2\theta+\cos^4\theta\).

Then

\(\cos5\theta=\cos^5\theta-10\cos^3\theta(1-\cos^2\theta)+5\cos\theta(1-2\cos^2\theta+\cos^4\theta)\).

Expanding and simplifying,

\(\cos5\theta=\cos^5\theta-10\cos^3\theta+10\cos^5\theta+5\cos\theta-10\cos^3\theta+5\cos^5\theta\)

\(=16\cos^5\theta-20\cos^3\theta+5\cos\theta\).

Hence

\(\boxed{\cos5\theta=16\cos^5\theta-20\cos^3\theta+5\cos\theta}.\)

(b) Let \(x=\cos\theta\). Then from part (a),

\(16x^5-20x^3+5x=\cos5\theta\).

So

\(32x^5-40x^3+10x-\sqrt2=0\)

becomes

\(2\cos5\theta=\sqrt2\),

hence

\(\cos5\theta=\frac{\sqrt2}{2}=\cos\frac{\pi}{4}.\)

Therefore

\(5\theta=\pm\frac{\pi}{4}+2k\pi.\)

So

\(\theta=\pm\frac{\pi}{20}+\frac{2k\pi}{5}.\)

Taking five distinct values gives

\(\theta=\frac{\pi}{20},\;\frac{9\pi}{20},\;\frac{17\pi}{20},\;\frac{25\pi}{20},\;\frac{33\pi}{20}.\)

Hence the roots are

\(\boxed{x=\cos\left(\frac{\pi}{20}\right),\;\cos\left(\frac{9\pi}{20}\right),\;\cos\left(\frac{17\pi}{20}\right),\;\cos\left(\frac{25\pi}{20}\right),\;\cos\left(\frac{33\pi}{20}\right)}.\)