Answer: Either

(i) \(\overrightarrow{AB}=(3-7,\,5-4,\,-2-(-1))=(-4,1,-1)\), so \(\overrightarrow{AB}=-1\) times \((4,-1,1)\). Also \(\overrightarrow{CD}=(2-2,\,7-6,\,\lambda-3)=(0,1,\lambda-3)\).

The shortest distance between the lines \(AB\) and \(CD\) is

\(\frac{|(\overrightarrow{AC})\cdot(\overrightarrow{AB}\times\overrightarrow{CD})|}{|\overrightarrow{AB}\times\overrightarrow{CD}|}=3\).

Now \(\overrightarrow{AC}=(2-7,\,6-4,\,3-(-1))=(-5,2,4)\), and

\(\overrightarrow{AB}\times\overrightarrow{CD}=(-4,1,-1)\times(0,1,\lambda-3)=(\lambda-2,\,4\lambda-12,\,-4)\).

Hence

\(\overrightarrow{AC}\cdot(\overrightarrow{AB}\times\overrightarrow{CD})=(-5,2,4)\cdot(\lambda-2,\,4\lambda-12,\,-4)=3\lambda-30\).

So

\(\frac{|3\lambda-30|}{\sqrt{(\lambda-2)^2+(4\lambda-12)^2+16}}=3\).

Squaring and simplifying gives \(\lambda^2-5\lambda+4=0\).

(ii) The values of \(\lambda\) are \(1\) and \(4\).

For \(\lambda=1\), a normal to plane \(ABD\) is

\(\mathbf{n}_1=\overrightarrow{AB}\times\overrightarrow{AD}=(4,-1,1)\times(-5,3,2)=(5,13,-7)\).

For \(\lambda=4\), a normal is

\(\mathbf{n}_2=(4,-1,1)\times(-5,3,5)=(8,25,-7)\).

The acute angle \(\theta\) between the planes satisfies

\(\cos\theta=\frac{|\mathbf{n}_1\cdot\mathbf{n}_2|}{|\mathbf{n}_1||\mathbf{n}_2|}\).

Now \(\mathbf{n}_1\cdot\mathbf{n}_2=5\cdot 8+13\cdot 25+(-7)(-7)=394\), so

\(\theta=\cos^{-1}\!\left(\frac{394}{\sqrt{243}\,\sqrt{738}}\right)\approx 12.1^\circ\).

Or

(i) Row-reducing the matrix gives an echelon form such as

\(\begin{pmatrix}1&2&-1&-1\\0&1&0&1\\0&0&1&1\\0&0&0&0\end{pmatrix}\).

There are 3 non-zero rows, so \(\dim(V)=3\).

(ii) If

\(\alpha_1\begin{pmatrix}1\\1\\1\\0\end{pmatrix}+\alpha_2\begin{pmatrix}2\\3\\0\\3\end{pmatrix}+\alpha_3\begin{pmatrix}-1\\-1\\3\\-4\end{pmatrix}=\mathbf{0}\),

then comparing components gives a linear system in \(\alpha_1,\alpha_2,\alpha_3\). Solving it yields only the trivial solution \(\alpha_1=\alpha_2=\alpha_3=0\), so the three vectors are linearly independent.

(iii) Since the range space has dimension 3, and the three given linearly independent vectors lie in \(V\), a basis is

\(\left\{\begin{pmatrix}1\\1\\1\\0\end{pmatrix},\begin{pmatrix}2\\3\\0\\3\end{pmatrix},\begin{pmatrix}-1\\-1\\3\\-4\end{pmatrix}\right\}\).

(iv) \(W\) is not a vector space, because it does not contain the zero vector. Since \(\mathbf{0}\\in V\), we have \(\mathbf{0}\notin W\).

(v) Let \(\begin{pmatrix}x\\y\\z\\t\end{pmatrix}\\in V\). Then there exist scalars \(\alpha,\beta,\gamma\) such that

\(\begin{pmatrix}x\\y\\z\\t\end{pmatrix}=\alpha\begin{pmatrix}1\\1\\1\\0\end{pmatrix}+\beta\begin{pmatrix}2\\3\\0\\3\end{pmatrix}+\gamma\begin{pmatrix}-1\\-1\\3\\-4\end{pmatrix}\).

Hence

\(y=\alpha+3\beta-\gamma,\quad z=\alpha+3\gamma,\quad t=3\beta-4\gamma\).

So

\(y-z-t=(\alpha+3\beta-\gamma)-(\alpha+3\gamma)-(3\beta-4\gamma)=0\).

Therefore every vector in \(V\) satisfies \(y-z-t=0\), so if \(\begin{pmatrix}x\\y\\z\\t\end{pmatrix}\in W\), then \(y-z-t\neq 0\).

Either

(i) First find the direction vectors of the two lines:

\(\overrightarrow{AB}=(3-7,\,5-4,\,-2-(-1))=(-4,1,-1)\),

\(\overrightarrow{CD}=(2-2,\,7-6,\,\lambda-3)=(0,1,\lambda-3)\).

The vector joining a point on \(AB\) to a point on \(CD\) can be taken as \(\overrightarrow{AC}\), where

\(\overrightarrow{AC}=(2-7,\,6-4,\,3-(-1))=(-5,2,4)\).

The shortest distance between skew lines is

\(d=\frac{|\overrightarrow{AC}\cdot(\overrightarrow{AB}\times\overrightarrow{CD})|}{|\overrightarrow{AB}\times\overrightarrow{CD}|}\).

Now

\(\overrightarrow{AB}\times\overrightarrow{CD}=(-4,1,-1)\times(0,1,\lambda-3)=(\lambda-2,\,4\lambda-12,\,-4)\).

Then

\(\overrightarrow{AC}\cdot(\overrightarrow{AB}\times\overrightarrow{CD})=(-5,2,4)\cdot(\lambda-2,\,4\lambda-12,\,-4)\)

\(=-5(\lambda-2)+2(4\lambda-12)-16=3\lambda-30\).

Given that the shortest distance is 3,

\(\frac{|3\lambda-30|}{\sqrt{(\lambda-2)^2+(4\lambda-12)^2+16}}=3\).

Square both sides:

\((3\lambda-30)^2=9\big((\lambda-2)^2+(4\lambda-12)^2+16\big)\).

Divide by 9 and expand:

\((\lambda-10)^2=(\lambda-2)^2+(4\lambda-12)^2+16\).

\(\lambda^2-20\lambda+100=(\lambda^2-4\lambda+4)+(16\lambda^2-96\lambda+144)+16\).

So

\(\lambda^2-20\lambda+100=17\lambda^2-100\lambda+164\),

hence

\(0=16\lambda^2-80\lambda+64\)

and therefore

\(\lambda^2-5\lambda+4=0\).

(ii) The roots are \(\lambda=1\) and \(\lambda=4\).

For the plane through \(A,B,D\), use normal vector \(\overrightarrow{AB}\times\overrightarrow{AD}\).

When \(\lambda=1\),

\(\overrightarrow{AD}=(2-7,\,7-4,\,1-(-1))=(-5,3,2)\),

so

\(\mathbf{n}_1=\overrightarrow{AB}\times\overrightarrow{AD}=(-4,1,-1)\times(-5,3,2)=(5,13,-7)\).

When \(\lambda=4\),

\(\overrightarrow{AD}=(-5,3,5)\),

so

\(\mathbf{n}_2=(-4,1,-1)\times(-5,3,5)=(8,25,-7)\).

The acute angle \(\theta\) between the planes is the acute angle between their normals:

\(\cos\theta=\frac{|\mathbf{n}_1\cdot\mathbf{n}_2|}{|\mathbf{n}_1||\mathbf{n}_2|}\).

Compute

\(\mathbf{n}_1\cdot\mathbf{n}_2=5\cdot 8+13\cdot 25+(-7)(-7)=394\),

\(|\mathbf{n}_1|=\sqrt{5^2+13^2+(-7)^2}=\sqrt{243}\),

\(|\mathbf{n}_2|=\sqrt{8^2+25^2+(-7)^2}=\sqrt{738}\).

Therefore

\(\cos\theta=\frac{394}{\sqrt{243}\sqrt{738}}\),

so

\(\theta\approx 12.1^\circ\).

Or

(i) Row-reduce the matrix:

\(\begin{pmatrix}1&2&-1&-1\\1&3&-1&0\\1&0&3&1\\0&3&-4&-1\end{pmatrix}\)

\(\to\begin{pmatrix}1&2&-1&-1\\0&1&0&1\\0&0&1&1\\0&0&0&0\end{pmatrix}\).

There are 3 pivots, so the range space has dimension \(3\).

(ii) To show the three vectors are linearly independent, suppose

\(\alpha_1\begin{pmatrix}1\\1\\1\\0\end{pmatrix}+\alpha_2\begin{pmatrix}2\\3\\0\\3\end{pmatrix}+\alpha_3\begin{pmatrix}-1\\-1\\3\\-4\end{pmatrix}=\mathbf{0}\).

Comparing components gives

\(\alpha_1+2\alpha_2-\alpha_3=0\),

\(\alpha_1+3\alpha_2-\alpha_3=0\),

\(\alpha_1+3\alpha_3=0\),

\(3\alpha_2-4\alpha_3=0\).

Subtracting the first two equations gives \(\alpha_2=0\), and then \(3\alpha_2-4\alpha_3=0\) gives \(\alpha_3=0\). Finally \(\alpha_1=0\). So only the trivial solution exists, and the vectors are linearly independent.

(iii) Since \(V\) has dimension 3 and these three independent vectors lie in \(V\), they form a basis:

\(\left\{\begin{pmatrix}1\\1\\1\\0\end{pmatrix},\begin{pmatrix}2\\3\\0\\3\end{pmatrix},\begin{pmatrix}-1\\-1\\3\\-4\end{pmatrix}\right\}\).

(iv) \(W\) is not a vector space, because \(W\) does not contain the zero vector. Indeed, \(\mathbf{0}\\in V\), so \(\mathbf{0}\notin W\).

(v) Since the vectors in part (iii) form a basis for \(V\), any vector \(\begin{pmatrix}x\\y\\z\\t\end{pmatrix}\in V\) can be written as

\(\alpha\begin{pmatrix}1\\1\\1\\0\end{pmatrix}+\beta\begin{pmatrix}2\\3\\0\\3\end{pmatrix}+\gamma\begin{pmatrix}-1\\-1\\3\\-4\end{pmatrix}\).

So

\(y=\alpha+3\beta-\gamma\), \(z=\alpha+3\gamma\), \(t=3\beta-4\gamma\).

Hence

\(y-z-t=(\alpha+3\beta-\gamma)-(\alpha+3\gamma)-(3\beta-4\gamma)=0\).

Therefore every vector in \(V\) satisfies \(y-z-t=0\). So if \(\begin{pmatrix}x\\y\\z\\t\end{pmatrix}\in W\), it cannot satisfy \(y-z-t=0\); equivalently, \(y-z-t\neq 0\).