Answer: The rank of \(\mathbf M\) is \(2\).
A basis for the null space is
\(\left\{\begin{pmatrix}1\\0\\1\\1\end{pmatrix},\begin{pmatrix}2\\1\\0\\0\end{pmatrix}\right\}.\)
\(\mathbf M\begin{pmatrix}1\\-2\\2\\-1\end{pmatrix}=\begin{pmatrix}15\\33\\66\\81\end{pmatrix}.\)
The required solution is
\(\mathbf x'=\begin{pmatrix}4\\-1\\3\\0\end{pmatrix}.\)
Row-reducing \(\mathbf M\) gives a row echelon form with two non-zero rows, for example
\(\begin{pmatrix}1&-2&3&-4\\0&0&1&-1\\0&0&0&0\\0&0&0&0\end{pmatrix}.\)
So \(\operatorname{rank}(\mathbf M)=2\).
For the null space, solve
\(x-2y+3z-4t=0,\qquad z-t=0.\)
Let \(z=t=\lambda\) and \(y=\mu\). Then \(x=2\mu+\lambda\), so
\(\mathbf x=\lambda\begin{pmatrix}1\\0\\1\\1\end{pmatrix}+\mu\begin{pmatrix}2\\1\\0\\0\end{pmatrix}.\)
A basis for \(K\) is therefore
\(\left\{\begin{pmatrix}1\\0\\1\\1\end{pmatrix},\begin{pmatrix}2\\1\\0\\0\end{pmatrix}\right\}.\)
Direct multiplication gives
\(\mathbf M\begin{pmatrix}1\\-2\\2\\-1\end{pmatrix}=\begin{pmatrix}15\\33\\66\\81\end{pmatrix}.\)
Hence the general solution is
\(\mathbf x=\begin{pmatrix}1\\-2\\2\\-1\end{pmatrix}+\lambda\begin{pmatrix}1\\0\\1\\1\end{pmatrix}+\mu\begin{pmatrix}2\\1\\0\\0\end{pmatrix}.\)
The sum of the components is \(6\), so
\(3\lambda+3\mu=6,\qquad \mu=2-\lambda.\)
The sum of the squares is \(26\), which gives
\(5\mu^2+4\lambda\mu+4\lambda+3\lambda^2+10=26.\)
Substitute \(\mu=2-\lambda\):
\(4\lambda^2-8\lambda+4=0,\)
so \(\lambda=1\), and hence \(\mu=1\). Therefore
\(\mathbf x'=\begin{pmatrix}1\\-2\\2\\-1\end{pmatrix}+\begin{pmatrix}1\\0\\1\\1\end{pmatrix}+\begin{pmatrix}2\\1\\0\\0\end{pmatrix}=\begin{pmatrix}4\\-1\\3\\0\end{pmatrix}.\)
