Answer: (a) \(c=\frac{1}{8}\), \(a=5\), \(b=1\).
(b) The median is \(2\sqrt{2}\).
(c) \(\mathrm{E}(\sqrt{X})=\frac{4}{9}+\frac{1}{2}\sqrt{6}\approx 1.67\).
(a) Since the upper quartile is 4, \(P(X\le 4)=\frac34\). Therefore the probability to the right of 4 is \(\frac14\).
For \(4\le x\le 6\), the density is constant, so
\(\displaystyle \int_4^6 c\,dx=\frac14\).
Thus \(2c=\frac14\), so \(c=\frac18\).
The probability on \([0,4]\) is \(\frac34\), hence
\(\displaystyle \int_0^4 \frac1{128}(4ax-bx^3)\,dx=\frac34\).
Integrating gives
\(\displaystyle \frac1{128}\left[2ax^2-\frac{b}{4}x^4\right]_0^4=\frac34\).
So
\(\displaystyle \frac1{128}(32a-64b)=\frac34\),
and therefore
\(32a-64b=96\), so \(a-2b=3\).
The density is continuous at \(x=4\), so
\(\displaystyle \frac1{128}(4a\cdot4-b\cdot4^3)=\frac18\).
This gives
\(16a-64b=16\), so \(a-4b=1\).
Solving \(a-2b=3\) and \(a-4b=1\): subtracting gives \(2b=2\), hence \(b=1\), and then \(a=5\).
Therefore \(c=\frac18\), \(a=5\), and \(b=1\).
(b) With \(a=5\) and \(b=1\), for \(0\le x\le4\),
\(\displaystyle f(x)=\frac1{128}(20x-x^3)\).
The median \(m\) lies in \([0,4]\), since \(P(X\le4)=\frac34\). So
\(\displaystyle \int_0^m \frac1{128}(20x-x^3)\,dx=\frac12\).
Integrating,
\(\displaystyle \frac1{128}\left(10m^2-\frac{m^4}{4}\right)=\frac12\).
Multiplying by 512 gives
\(40m^2-m^4=256\),
so
\(m^4-40m^2+256=0\).
Let \(y=m^2\). Then
\(y^2-40y+256=0\).
Hence
\(\displaystyle y=\frac{40\pm\sqrt{1600-1024}}{2}=\frac{40\pm24}{2}\).
So \(y=32\) or \(y=8\). Since \(0\le m\le4\), we take \(m^2=8\), giving
\(m=2\sqrt2\).
(c) We need
\(\displaystyle \mathrm{E}(\sqrt X)=\frac1{128}\int_0^4 \sqrt{x}(20x-x^3)\,dx+\frac18\int_4^6 \sqrt{x}\,dx\).
So
\(\displaystyle \mathrm{E}(\sqrt X)=\frac1{128}\int_0^4 (20x^{3/2}-x^{7/2})\,dx+\frac18\int_4^6 x^{1/2}\,dx\).
Integrating,
\(\displaystyle \mathrm{E}(\sqrt X)=\frac1{128}\left[8x^{5/2}-\frac{2}{9}x^{9/2}\right]_0^4+\frac1{12}\left[x^{3/2}\right]_4^6\).
For the first part,
\(\displaystyle \frac1{128}\left(8\cdot4^{5/2}-\frac{2}{9}4^{9/2}\right)=\frac1{128}\left(256-\frac{1024}{9}\right)=\frac{10}{9}\).
For the second part,
\(\displaystyle \frac1{12}(6^{3/2}-4^{3/2})=\frac1{12}(6\sqrt6-8)=\frac12\sqrt6-\frac23\).
Therefore
\(\displaystyle \mathrm{E}(\sqrt X)=\frac{10}{9}+\frac12\sqrt6-\frac23=\frac49+\frac12\sqrt6\).
So, to 2 decimal places,
\(\mathrm{E}(\sqrt X)=\frac49+\frac12\sqrt6\approx1.67\).
