Answer: (a) \(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(\frac{x}{2}\sqrt{x^2-9}-\frac{9}{2}\cosh^{-1}\frac{x}{3}\right)=\sqrt{x^2-9}.\)

(b) \(\displaystyle y=x\left(\frac{x}{2}\sqrt{x^2-9}-\frac{9}{2}\cosh^{-1}\frac{x}{3}+\frac13\right).\)

(a)

Differentiate term by term.

Using the product rule,

\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(\frac{x}{2}\sqrt{x^2-9}\right)=\frac12\sqrt{x^2-9}+\frac{x}{2}\cdot\frac{1}{2\sqrt{x^2-9}}\cdot 2x\)

so

\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(\frac{x}{2}\sqrt{x^2-9}\right)=\frac12\sqrt{x^2-9}+\frac{x^2}{2\sqrt{x^2-9}}.\)

Also,

\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(\cosh^{-1}\frac{x}{3}\right)=\frac{1}{\sqrt{x^2-9}}.\)

Therefore

\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(-\frac92\cosh^{-1}\frac{x}{3}\right)=-\frac{9}{2\sqrt{x^2-9}}.\)

Hence

\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(\frac{x}{2}\sqrt{x^2-9}-\frac92\cosh^{-1}\frac{x}{3}\right)=\frac{x^2}{2\sqrt{x^2-9}}+\frac12\sqrt{x^2-9}-\frac{9}{2\sqrt{x^2-9}}.\)

Combine the fractional terms:

\(\displaystyle =\frac{x^2-9}{2\sqrt{x^2-9}}+\frac12\sqrt{x^2-9}=\frac12\sqrt{x^2-9}+\frac12\sqrt{x^2-9}=\sqrt{x^2-9}.\)

This proves the result.

(b)

The differential equation is

\(\displaystyle x\frac{\mathrm{d}y}{\mathrm{d}x}-y=x^2\sqrt{x^2-9}.\)

Divide through by \(x\):

\(\displaystyle \frac{\mathrm{d}y}{\mathrm{d}x}-\frac{y}{x}=x\sqrt{x^2-9}.\)

This is linear, with integrating factor

\(\displaystyle \mathrm{IF}=e^{\int -1/x\,\mathrm{d}x}=e^{-\ln x}=x^{-1}.\)

Multiply through by \(x^{-1}\):

\(\displaystyle x^{-1}\frac{\mathrm{d}y}{\mathrm{d}x}-x^{-2}y=\sqrt{x^2-9}.\)

The left-hand side is

\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(x^{-1}y\right),\)

so

\(\displaystyle \frac{\mathrm{d}}{\mathrm{d}x}\left(x^{-1}y\right)=\sqrt{x^2-9}.\)

Integrating,

\(\displaystyle x^{-1}y=\int \sqrt{x^2-9}\,\mathrm{d}x=\frac{x}{2}\sqrt{x^2-9}-\frac92\cosh^{-1}\frac{x}{3}+C.\)

Use \(y=1\) when \(x=3\):

\(\displaystyle \frac13=\frac32\sqrt{9-9}-\frac92\cosh^{-1}(1)+C.\)

Since \(\sqrt{0}=0\) and \(\cosh^{-1}(1)=0\),

\(\displaystyle C=\frac13.\)

Therefore

\(\displaystyle x^{-1}y=\frac{x}{2}\sqrt{x^2-9}-\frac92\cosh^{-1}\frac{x}{3}+\frac13.\)

Multiplying by \(x\),

\(\displaystyle y=x\left(\frac{x}{2}\sqrt{x^2-9}-\frac92\cosh^{-1}\frac{x}{3}+\frac13\right).\)