Answer: (a) \(\int_{0}^{\pi/2} x\sin x\,\mathrm{d}x = 1\).

(b) For \(I_n=\int_{0}^{\pi/2} x^n\sin x\,\mathrm{d}x\), \(I_n = n\left(\frac{\pi}{2}\right)^{n-1}-n(n-1)I_{n-2}\) for \(n\gt 1\).

(c) \(\int_{0}^{1}(\cos^{-1}u)^3\,\mathrm{d}u = \frac{3\pi^2}{4}-6\).

(a) Use integration by parts with \(u=x\) and \(\mathrm{d}v=\sin x\,\mathrm{d}x\). Then \(\mathrm{d}u=\mathrm{d}x\) and \(v=-\cos x\), so

\(\int_{0}^{\pi/2} x\sin x\,\mathrm{d}x = \left[-x\cos x\right]_{0}^{\pi/2} + \int_{0}^{\pi/2} \cos x\,\mathrm{d}x.\)

Now \(\left[-x\cos x\right]_{0}^{\pi/2}=0\) and \(\int_{0}^{\pi/2} \cos x\,\mathrm{d}x = \left[\sin x\right]_{0}^{\pi/2}=1\). Hence

\(\int_{0}^{\pi/2} x\sin x\,\mathrm{d}x=1.\)

(b) Let \(I_n=\int_{0}^{\pi/2} x^n\sin x\,\mathrm{d}x\). Integrate by parts with \(u=x^n\), \(\mathrm{d}v=\sin x\,\mathrm{d}x\). Then \(\mathrm{d}u=n x^{n-1}\,\mathrm{d}x\) and \(v=-\cos x\), giving

\(I_n=\left[-x^n\cos x\right]_{0}^{\pi/2}+\int_{0}^{\pi/2} n x^{n-1}\cos x\,\mathrm{d}x.\)

Now integrate the remaining integral by parts again, taking \(u= n x^{n-1}\), \(\mathrm{d}v=\cos x\,\mathrm{d}x\). Then \(\mathrm{d}u=n(n-1)x^{n-2}\,\mathrm{d}x\) and \(v=\sin x\), so

\(\int_{0}^{\pi/2} n x^{n-1}\cos x\,\mathrm{d}x = \left[nx^{n-1}\sin x\right]_{0}^{\pi/2}-\int_{0}^{\pi/2} n(n-1)x^{n-2}\sin x\,\mathrm{d}x.\)

Therefore

\(I_n=\left[-x^n\cos x + nx^{n-1}\sin x\right]_{0}^{\pi/2} - n(n-1)\int_{0}^{\pi/2} x^{n-2}\sin x\,\mathrm{d}x.\)

The bracketed term is \(n\left(\frac{\pi}{2}\right)^{n-1}\), and the remaining integral is \(I_{n-2}\). So

\(I_n=n\left(\frac{\pi}{2}\right)^{n-1}-n(n-1)I_{n-2}.\)

(c) Use the substitution \(x=\cos^{-1}u\). Then \(u=\cos x\) and \(\mathrm{d}u=-\sin x\,\mathrm{d}x\). Also, when \(u=0\), \(x=\pi/2\); when \(u=1\), \(x=0\). Hence

\(\int_{0}^{1}(\cos^{-1}u)^3\,\mathrm{d}u = \int_{\pi/2}^{0} x^3(-\sin x)\,\mathrm{d}x = \int_{0}^{\pi/2} x^3\sin x\,\mathrm{d}x = I_3.\)

Now apply the recurrence with \(n=3\):

\(I_3 = 3\left(\frac{\pi}{2}\right)^2 - 3\cdot 2\, I_1.\)

From part (a), \(I_1=1\). Therefore

\(I_3 = 3\cdot \frac{\pi^2}{4} - 6 = \frac{3\pi^2}{4}-6.\)

So \(\int_{0}^{1}(\cos^{-1}u)^3\,\mathrm{d}u = \frac{3\pi^2}{4}-6.\)