Answer: The corresponding eigenvectors can be taken as

• for eigenvalue \(1\): \(\begin{pmatrix}17\\-6\\-7\end{pmatrix}\)
• for eigenvalue \(5\): \(\begin{pmatrix}1\\-2\\1\end{pmatrix}\)
• for eigenvalue \(7\): \(\begin{pmatrix}1\\0\\1\end{pmatrix}\)

So one suitable choice is

\(\mathbf{P}=\begin{pmatrix}17 & 1 & 1\\ -6 & -2 & 0\\ -7 & 1 & 1\end{pmatrix}\), \(\mathbf{D}=\begin{pmatrix}1^n & 0 & 0\\ 0 & 5^n & 0\\ 0 & 0 & 7^n\end{pmatrix}\).

The values of \(k\) for which \(k^n\mathbf{A}^n\) tends to the zero matrix are \(-\frac{1}{7} \lt k \lt \frac{1}{7}\).

Let \(\lambda\) be an eigenvalue of \(\mathbf{A}\). We find an eigenvector by solving \((\mathbf{A}-\lambda\mathbf{I})\mathbf{x}=\mathbf{0}\).

For \(\lambda=1\),

\(\mathbf{A}-\mathbf{I}=\begin{pmatrix}2&1&4\\1&4&-1\\2&1&4\end{pmatrix}\).

Solving \(\begin{pmatrix}2&1&4\\1&4&-1\\2&1&4\end{pmatrix}\begin{pmatrix}x\\y\\z\end{pmatrix}=\mathbf{0}\) gives an eigenvector proportional to \(\begin{pmatrix}17\\-6\\-7\end{pmatrix}\).

For \(\lambda=5\),

\(\mathbf{A}-5\mathbf{I}=\begin{pmatrix}-2&1&4\\1&0&-1\\2&1&0\end{pmatrix}\).

Solving \((\mathbf{A}-5\mathbf{I})\mathbf{x}=\mathbf{0}\) gives an eigenvector proportional to \(\begin{pmatrix}1\\-2\\1\end{pmatrix}\).

For \(\lambda=7\),

\(\mathbf{A}-7\mathbf{I}=\begin{pmatrix}-4&1&4\\1&-2&-1\\2&1&-2\end{pmatrix}\).

Solving \((\mathbf{A}-7\mathbf{I})\mathbf{x}=\mathbf{0}\) gives an eigenvector proportional to \(\begin{pmatrix}1\\0\\1\end{pmatrix}\).

Hence a corresponding set of eigenvectors is

\(\begin{pmatrix}17\\-6\\-7\end{pmatrix},\;\begin{pmatrix}1\\-2\\1\end{pmatrix},\;\begin{pmatrix}1\\0\\1\end{pmatrix}.\)

Using these as columns, we may take

\(\mathbf{P}=\begin{pmatrix}17 & 1 & 1\\ -6 & -2 & 0\\ -7 & 1 & 1\end{pmatrix}.\)

Since the eigenvalues of \(\mathbf{A}\) are \(1,5,7\), the eigenvalues of \(\mathbf{A}^n\) are \(1^n,5^n,7^n\). Therefore

\(\mathbf{A}^n=\mathbf{PDP}^{-1}\), where

\(\mathbf{D}=\begin{pmatrix}1^n&0&0\\0&5^n&0\\0&0&7^n\end{pmatrix}.\)

Now consider \(k^n\mathbf{A}^n\):

\(k^n\mathbf{A}^n=\mathbf{P}\begin{pmatrix}k^n&0&0\\0&(5k)^n&0\\0&0&(7k)^n\end{pmatrix}\mathbf{P}^{-1}.\)

For this to tend to the zero matrix as \(n\to\infty\), each diagonal factor must tend to \(0\). This requires \(|k|\lt 1\), \(|5k|\lt 1\), and \(|7k|\lt 1\). The strongest condition is \(|7k|\lt 1\), so

\(-\frac{1}{7} \lt k \lt \frac{1}{7}.\)