(a) The transformation matrix for a stretch parallel to the x-axis with scale factor k is \(\begin{pmatrix} k & 0 \\ 0 & 1 \end{pmatrix}\). The shear transformation with the x-axis fixed and (0, 1) mapped to (k, 1) is \(\begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}\). The combined transformation matrix M is the product:

\(\mathbf{M} = \begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix} \begin{pmatrix} k & 0 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} k & k \\ 0 & 1 \end{pmatrix}\)

(b) The transformation \(\mathbf{M} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} kx + ky \\ y \end{pmatrix}\) implies invariant points satisfy \(kx + ky = x\). Solving gives \(y = \frac{1-k}{k}x\).

(c) The inverse matrix \(\mathbf{M}^{-1}\) is found by inverting \(\mathbf{M}\):

\(\mathbf{M}^{-1} = \frac{1}{k} \begin{pmatrix} 1 & -k \\ 0 & k \end{pmatrix}\)

(d) The area of P is given by the determinant of M, \(|k| = 3k^2\). Solving \(|k| = 3k^2\) gives \(k = \pm \frac{1}{3}\).