Answer: (a) The eigenvalues of \(\mathbf{A}\) are \(1\), \(2\) and \(-3\).

(b) \(\mathbf{A}^{-1}=p\mathbf{A}^2+q\mathbf{I}\) with

\(p=-\frac{1}{6}\), \(q=\frac{7}{6}\).

So \(\mathbf{A}^{-1}=-\frac{1}{6}\mathbf{A}^2+\frac{7}{6}\mathbf{I}\).

(a) The eigenvalues are found from the characteristic equation

\(\det(\mathbf{A}-\lambda \mathbf{I})=0\).

Thus

\(\left|\begin{array}{ccc}6-\lambda & -9 & 5 \\ 5 & -8-\lambda & 5 \\ 1 & -1 & 2-\lambda\end{array}\right|=0\).

Expanding gives

\(\lambda^3-7\lambda+6=0\).

Factorising,

\(\lambda^3-7\lambda+6=(\lambda-1)(\lambda-2)(\lambda+3)\).

Hence the eigenvalues are

\(\lambda=1,\;2,\;-3\).

(b) Using the characteristic equation from part (a), and applying it to \(\mathbf{A}\),

\(\mathbf{A}^3-7\mathbf{A}+6\mathbf{I}=\mathbf{0}\).

Since none of the eigenvalues is \(0\), the matrix \(\mathbf{A}\) is invertible. Multiplying through by \(\mathbf{A}^{-1}\) gives

\(\mathbf{A}^2-7\mathbf{I}+6\mathbf{A}^{-1}=\mathbf{0}\).

So

\(6\mathbf{A}^{-1}=-\mathbf{A}^2+7\mathbf{I}\),

and therefore

\(\mathbf{A}^{-1}=-\frac{1}{6}\mathbf{A}^2+\frac{7}{6}\mathbf{I}\).

Thus \(p=-\frac{1}{6}\) and \(q=\frac{7}{6}\).