Answer: (a) \(\cot 6\theta=\dfrac{\cot ^6\theta-15\cot ^4\theta+15\cot ^2\theta-1}{6\cot ^5\theta-20\cot ^3\theta+6\cot \theta}.\)
(b) The roots are
\(x=\cot\left(\frac{\pi}{24}\right),\;\cot\left(\frac{5\pi}{24}\right),\;\cot\left(\frac{3\pi}{8}\right),\;\cot\left(\frac{13\pi}{24}\right),\;\cot\left(\frac{17\pi}{24}\right),\;\cot\left(\frac{7\pi}{8}\right).\)
(a) By de Moivre's theorem,
\((\cos\theta+i\sin\theta)^6=\cos 6\theta+i\sin 6\theta.\)
Expanding,
\((\cos\theta+i\sin\theta)^6=\cos^6\theta+6i\cos^5\theta\sin\theta-15\cos^4\theta\sin^2\theta-20i\cos^3\theta\sin^3\theta+15\cos^2\theta\sin^4\theta+6i\cos\theta\sin^5\theta-\sin^6\theta.\)
Equating real and imaginary parts gives
\(\cos 6\theta=\cos^6\theta-15\cos^4\theta\sin^2\theta+15\cos^2\theta\sin^4\theta-\sin^6\theta,\)
\(\sin 6\theta=6\cos^5\theta\sin\theta-20\cos^3\theta\sin^3\theta+6\cos\theta\sin^5\theta.\)
Hence
\(\cot 6\theta=\dfrac{\cos 6\theta}{\sin 6\theta}=\dfrac{\cos^6\theta-15\cos^4\theta\sin^2\theta+15\cos^2\theta\sin^4\theta-\sin^6\theta}{6\cos^5\theta\sin\theta-20\cos^3\theta\sin^3\theta+6\cos\theta\sin^5\theta}.\)
Now divide numerator and denominator by \(\sin^6\theta\):
\(\cot 6\theta=\dfrac{\cot^6\theta-15\cot^4\theta+15\cot^2\theta-1}{6\cot^5\theta-20\cot^3\theta+6\cot\theta}.\)
(b) Let \(x=\cot\theta\). Then
\(x^6-6x^5-15x^4+20x^3+15x^2-6x-1=0\)
becomes
\(x^6-15x^4+15x^2-1=6x^5-20x^3+6x,\)
so
\(\dfrac{x^6-15x^4+15x^2-1}{6x^5-20x^3+6x}=1.\)
Using the result from part (a), this is
\(\cot 6\theta=1.\)
Therefore
\(6\theta=\frac{\pi}{4}+k\pi\)
for integer \(k\), so
\(\theta=\frac{\pi}{24}+\frac{k\pi}{6}.\)
Taking six distinct values, for \(k=0,1,2,3,4,5\), gives
\(\theta=\frac{\pi}{24},\;\frac{5\pi}{24},\;\frac{3\pi}{8},\;\frac{13\pi}{24},\;\frac{17\pi}{24},\;\frac{7\pi}{8}.\)
Hence the roots are
\(x=\cot\left(\frac{\pi}{24}\right),\;\cot\left(\frac{5\pi}{24}\right),\;\cot\left(\frac{3\pi}{8}\right),\;\cot\left(\frac{13\pi}{24}\right),\;\cot\left(\frac{17\pi}{24}\right),\;\cot\left(\frac{7\pi}{8}\right).\)
