Answer: (a) \(G_X(t)=\frac{1}{5}t+\frac{3}{5}t^2+\frac{1}{5}t^3\).

(b) \(G_Y(t)=\frac{9}{16}+\frac{6}{16}t+\frac{1}{16}t^2\).

(c) \(G_Z(t)=G_X(t)G_Y(t)=\frac{9}{80}t+\frac{33}{80}t^2+\frac{28}{80}t^3+\frac{9}{80}t^4+\frac{1}{80}t^5\).

(d) \(\mathrm{E}(Z)=\frac{5}{2}\) and \(\mathrm{Var}(Z)=\frac{31}{40}\).

(a) Let \(X\) be the number of red balls in 3 balls chosen from 4 red and 2 blue.

Since there are only 2 blue balls, it is impossible to choose 0 red balls. Using combinations,

\(P(X=1)=\dfrac{\binom41\binom22}{\binom63}=\dfrac{4}{20}=\frac{1}{5}\),

\(P(X=2)=\dfrac{\binom42\binom21}{\binom63}=\dfrac{12}{20}=\frac{3}{5}\),

\(P(X=3)=\dfrac{\binom43\binom20}{\binom63}=\dfrac{4}{20}=\frac{1}{5}\).

So the probability generating function is

\(G_X(t)=\sum P(X=r)t^r=\frac{1}{5}t+\frac{3}{5}t^2+\frac{1}{5}t^3\).

(b) Let \(Y\) be the number of heads when two independent coins are thrown, with \(P(H)=\frac14\) and \(P(T)=\frac34\).

Then

\(P(Y=0)=\left(\frac34\right)^2=\frac{9}{16}\),

\(P(Y=1)=2\cdot \frac14\cdot \frac34=\frac{6}{16}\),

\(P(Y=2)=\left(\frac14\right)^2=\frac{1}{16}\).

Hence

\(G_Y(t)=\frac{9}{16}+\frac{6}{16}t+\frac{1}{16}t^2\).

(c) Since \(Z=X+Y\) and the ball selection and coin throws are independent,

\(G_Z(t)=G_X(t)G_Y(t)\).

So

\(G_Z(t)=\left(\frac{1}{5}t+\frac{3}{5}t^2+\frac{1}{5}t^3\right)\left(\frac{9}{16}+\frac{6}{16}t+\frac{1}{16}t^2\right).\)

Now expand:

from \(\frac15 t\): \(\frac{9}{80}t+\frac{6}{80}t^2+\frac{1}{80}t^3\),

from \(\frac35 t^2\): \(\frac{27}{80}t^2+\frac{18}{80}t^3+\frac{3}{80}t^4\),

from \(\frac15 t^3\): \(\frac{9}{80}t^3+\frac{6}{80}t^4+\frac{1}{80}t^5\).

Collecting like powers,

\(G_Z(t)=\frac{9}{80}t+\frac{33}{80}t^2+\frac{28}{80}t^3+\frac{9}{80}t^4+\frac{1}{80}t^5\).

(d) For a probability generating function \(G_Z(t)\),

\(\mathrm{E}(Z)=G_Z'(1)\),

and

\(\mathrm{Var}(Z)=G_Z''(1)+G_Z'(1)-\left(G_Z'(1)\right)^2\).

Differentiate:

\(G_Z'(t)=\frac{9}{80}+\frac{66}{80}t+\frac{84}{80}t^2+\frac{36}{80}t^3+\frac{5}{80}t^4\).

So

\(G_Z'(1)=\frac{9+66+84+36+5}{80}=\frac{200}{80}=\frac52\).

Therefore \(\mathrm{E}(Z)=\frac52\).

Differentiate again:

\(G_Z''(t)=\frac{66}{80}+\frac{168}{80}t+\frac{108}{80}t^2+\frac{20}{80}t^3\).

Hence

\(G_Z''(1)=\frac{66+168+108+20}{80}=\frac{362}{80}\).

Now

\(\mathrm{Var}(Z)=\frac{362}{80}+\frac{200}{80}-\left(\frac52\right)^2\)

\(=\frac{562}{80}-\frac{25}{4}=\frac{281}{40}-\frac{250}{40}=\frac{31}{40}.\)

So \(\mathrm{Var}(Z)=\frac{31}{40}\).