Answer: (a) \(\displaystyle \int \sin \theta\cos^n\theta\,d\theta=-\frac{\cos^{n+1}\theta}{n+1}+C\), for \(n\neq -1\).

(b) \(\displaystyle I_{m,n}=\frac{m-1}{m+n}I_{m-2,n}\).

(c) \(\displaystyle \cos^5\theta=\frac1{16}\cos5\theta+\frac5{16}\cos3\theta+\frac58\cos\theta\), so \(a=\frac1{16},\ b=\frac5{16},\ c=\frac58\).

(d) \(\displaystyle I_{2,5}=\frac{8}{105}\).

(a) Let \(u=\cos\theta\). Then \(du=-\sin\theta\,d\theta\), so

\(\displaystyle \int \sin\theta\cos^n\theta\,d\theta=-\int u^n\,du=-\frac{u^{n+1}}{n+1}+C\).

Hence

\(\displaystyle \int \sin \theta\cos^n\theta\,d\theta=-\frac{\cos^{n+1}\theta}{n+1}+C\), for \(n\neq -1\).

(b) Start with

\(\displaystyle I_{m,n}=\int_0^{\pi/2}\sin^m\theta\cos^n\theta\,d\theta=\int_0^{\pi/2}\sin^{m-1}\theta\,(\sin\theta\cos^n\theta)\,d\theta.\)

Integrate by parts with

\(u=\sin^{m-1}\theta\), so \(du=(m-1)\sin^{m-2}\theta\cos\theta\,d\theta\),

and

\(dv=\sin\theta\cos^n\theta\,d\theta\), so from part (a), \(v=-\dfrac{\cos^{n+1}\theta}{n+1}\).

Then

\(\displaystyle I_{m,n}=\left[-\frac{\sin^{m-1}\theta\cos^{n+1}\theta}{n+1}\right]_0^{\pi/2}+\frac{m-1}{n+1}\int_0^{\pi/2}\sin^{m-2}\theta\cos^{n+2}\theta\,d\theta.\)

The boundary term is zero, so

\(\displaystyle I_{m,n}=\frac{m-1}{n+1}\int_0^{\pi/2}\sin^{m-2}\theta\cos^n\theta\cos^2\theta\,d\theta.\)

Now use \(\cos^2\theta=1-\sin^2\theta\):

\(\displaystyle I_{m,n}=\frac{m-1}{n+1}\int_0^{\pi/2}\sin^{m-2}\theta\cos^n\theta(1-\sin^2\theta)\,d\theta.\)

So

\(\displaystyle I_{m,n}=\frac{m-1}{n+1}\left(\int_0^{\pi/2}\sin^{m-2}\theta\cos^n\theta\,d\theta-\int_0^{\pi/2}\sin^m\theta\cos^n\theta\,d\theta\right).\)

That is,

\(\displaystyle I_{m,n}=\frac{m-1}{n+1}I_{m-2,n}-\frac{m-1}{n+1}I_{m,n}.\)

Hence

\(\displaystyle \frac{m+n}{n+1}I_{m,n}=\frac{m-1}{n+1}I_{m-2,n},\)

so

\(\displaystyle I_{m,n}=\frac{m-1}{m+n}I_{m-2,n}.\)

(c) Since \(z=\cos\theta+i\sin\theta\), we have \(z^{-1}=\cos\theta-i\sin\theta\), so

\(\displaystyle z+z^{-1}=2\cos\theta.\)

Now expand and group:

\(\displaystyle \left(z+z^{-1}\right)^5=z^5+5z^3+10z+10z^{-1}+5z^{-3}+z^{-5}\)

\(\displaystyle \phantom{\left(z+z^{-1}\right)^5}=(z^5+z^{-5})+5(z^3+z^{-3})+10(z+z^{-1}).\)

By de Moivre's theorem, \(z^r+z^{-r}=2\cos r\theta\). Therefore

\(\displaystyle (2\cos\theta)^5=2\cos5\theta+5(2\cos3\theta)+10(2\cos\theta).\)

So

\(\displaystyle 32\cos^5\theta=2\cos5\theta+10\cos3\theta+20\cos\theta.\)

Dividing by \(32\),

\(\displaystyle \cos^5\theta=\frac1{16}\cos5\theta+\frac5{16}\cos3\theta+\frac58\cos\theta.\)

Thus \(a=\frac1{16}\), \(b=\frac5{16}\), \(c=\frac58\).

(d) Using the reduction formula with \(m=2\) and \(n=5\),

\(\displaystyle I_{2,5}=\frac{2-1}{2+5}I_{0,5}=\frac17 I_{0,5}.\)

Now

\(\displaystyle I_{0,5}=\int_0^{\pi/2}\cos^5\theta\,d\theta.\)

Using part (c),

\(\displaystyle I_{0,5}=\int_0^{\pi/2}\left(\frac1{16}\cos5\theta+\frac5{16}\cos3\theta+\frac58\cos\theta\right)d\theta.\)

Therefore

\(\displaystyle I_{0,5}=\left[\frac1{80}\sin5\theta+\frac5{48}\sin3\theta+\frac58\sin\theta\right]_0^{\pi/2}.\)

At \(\theta=\pi/2\),

\(\displaystyle \sin\frac{5\pi}{2}=1,\quad \sin\frac{3\pi}{2}=-1,\quad \sin\frac{\pi}{2}=1.\)

So

\(\displaystyle I_{0,5}=\frac1{80}-\frac5{48}+\frac58=\frac{8}{15}.\)

Hence

\(\displaystyle I_{2,5}=\frac17\cdot\frac{8}{15}=\frac{8}{105}.\)