Answer: (i) The shortest distance between the lines is \(\frac{6}{\sqrt{19}}\).

(ii) The required plane is \(19x+18y+13z=0\).

Let the position vectors of the points be

\(A=(1,-1,0),\quad B=(2,1,7),\quad C=(1,-1,1).\)

The line \(OC\) has direction vector

\(\mathbf{d}_1=(1,-1,1).\)

The line \(AB\) has direction vector

\(\mathbf{d}_2=B-A=(1,2,7).\)

Also, since \(O\) lies on \(OC\), a point on \(OC\) is \(O=(0,0,0)\), and a point on \(AB\) is \(A=(1,-1,0)\).

(i) Shortest distance between the lines

For two skew lines with direction vectors \(\mathbf{d}_1\) and \(\mathbf{d}_2\), the shortest distance is

\(\displaystyle \frac{|(\mathbf{a}-\mathbf{o})\cdot(\mathbf{d}_1\times \mathbf{d}_2)|}{|\mathbf{d}_1\times \mathbf{d}_2|}\),

where \(\mathbf{o}\) and \(\mathbf{a}\) are position vectors of one point on each line.

Here \(\mathbf{a}-\mathbf{o}=A-O=(1,-1,0)\).

Now

\(\mathbf{d}_1\times \mathbf{d}_2=\begin{vmatrix} \mathbf{i}&\mathbf{j}&\mathbf{k}\\ 1&-1&1\\ 1&2&7\end{vmatrix}=(-9,-6,3).\)

So

\(|\mathbf{d}_1\times \mathbf{d}_2|=\sqrt{(-9)^2+(-6)^2+3^2}=\sqrt{126}=3\sqrt{14}.\)

Also

\((1,-1,0)\cdot(-9,-6,3)=-9+6=-3,\)

so the absolute value is \(3\).

Hence the shortest distance is

\(\displaystyle \frac{3}{3\sqrt{14}}=\frac{1}{\sqrt{14}}.\)

(ii) Cartesian equation of the plane

The plane contains the line \(OC\), so it contains the vector \((1,-1,1)\). It also contains the common perpendicular of \(OC\) and \(AB\), so it must also contain a direction perpendicular to both lines. That common perpendicular has direction \(\mathbf{d}_1\times\mathbf{d}_2\), so the plane contains both \(\mathbf{d}_1\) and \(\mathbf{d}_1\times\mathbf{d}_2\).

Therefore a normal vector to the plane is perpendicular to both of these vectors. We can take

\(\mathbf{n}=\mathbf{d}_1\times(\mathbf{d}_1\times\mathbf{d}_2).\)

Using \(\mathbf{d}_1=(1,-1,1)\) and \(\mathbf{d}_1\times\mathbf{d}_2=(-9,-6,3)\),

\(\mathbf{n}=\begin{vmatrix} \mathbf{i}&\mathbf{j}&\mathbf{k}\\ 1&-1&1\\ -9&-6&3\end{vmatrix}=(3,-12,-15).\)

So a simpler normal vector is \((1,-4,-5)\).

The plane passes through \(O\), so its equation is

\(x-4y-5z=0.\)

But this is not the only possibility if we have chosen the wrong orientation? Let's verify by checking that the plane contains \(OC\): substituting \((1,-1,1)\) gives \(1+4-5=0\), so it does contain \(OC\). It also contains the common perpendicular direction \((-9,-6,3)\): \(-9+24-15=0\), so it is correct.