Answer: \((2n+1)I_n=\sqrt3-nI_{n-1}\), and \(I_3=\frac{2}{35}(\sqrt3+1)\).
Use integration by parts with \(u=x^n\) and \(\mathrm{d}v=(1+2x)^{-1/2}\,\mathrm{d}x\).
Then \(\mathrm{d}u=nx^{n-1}\,\mathrm{d}x\) and \(v=\sqrt{1+2x}\).
Hence
\(I_n=\left[x^n\sqrt{1+2x}\right]_0^1-n\int_0^1 x^{n-1}\sqrt{1+2x}\,\mathrm{d}x\).
The boundary term is \(\sqrt3\), so
\(I_n=\sqrt3-n\int_0^1 \frac{x^{n-1}(1+2x)}{\sqrt{1+2x}}\,\mathrm{d}x\).
Split the integral:
\(I_n=\sqrt3-n\int_0^1 \frac{x^{n-1}}{\sqrt{1+2x}}\,\mathrm{d}x-2n\int_0^1 \frac{x^n}{\sqrt{1+2x}}\,\mathrm{d}x\).
Therefore
\(I_n=\sqrt3-nI_{n-1}-2nI_n\).
Rearranging gives
\((2n+1)I_n=\sqrt3-nI_{n-1}\), as required.
Now
\(I_0=\int_0^1 \frac{1}{\sqrt{1+2x}}\,\mathrm{d}x=\left[\sqrt{1+2x}\right]_0^1=\sqrt3-1\).
For \(n=1\),
\(3I_1=\sqrt3-I_0=\sqrt3-(\sqrt3-1)=1\), so \(I_1=\frac13\).
For \(n=2\),
\(5I_2=\sqrt3-2I_1=\sqrt3-\frac23\), so \(I_2=\frac{\sqrt3}{5}-\frac{2}{15}\).
For \(n=3\),
\(7I_3=\sqrt3-3I_2=\sqrt3-3\left(\frac{\sqrt3}{5}-\frac{2}{15}\right)\).
Thus
\(7I_3=\sqrt3-\frac{3\sqrt3}{5}+\frac25=\frac{2\sqrt3}{5}+\frac25=\frac{2(\sqrt3+1)}{5}\).
Therefore
\(I_3=\frac{2}{35}(\sqrt3+1)\).
