Answer: (a) \(\left(\frac45,10\right)\) and \((-2,-4)\); (b) \(\left(\frac{12}{5},\frac{12}{5}\right)\).

(a) Since \(y=5x+6\) and \(xy=8\), substitute \(y=5x+6\) into \(xy=8\):

\(x(5x+6)=8.\)

So

\(5x^2+6x-8=0.\)

Factorise:

\((5x-4)(x+2)=0.\)

Hence

\(x=\frac45\quad\text{or}\quad x=-2.\)

When \(x=\frac45\),

\(y=5\left(\frac45\right)+6=10.\)

When \(x=-2\),

\(y=5(-2)+6=-4.\)

Therefore the points are

\(\left(\frac45,10\right)\quad\text{and}\quad(-2,-4).\)

(b) The midpoint of \(AB\) is

\(\left(\frac{\frac45+(-2)}2,\frac{10+(-4)}2\right) =\left(-\frac35,3\right).\)

The gradient of \(AB\) is the gradient of the line \(y=5x+6\), so it is \(5\).

Therefore the perpendicular bisector has gradient \(-\frac15\).

Its equation is

\(y-3=-\frac15\left(x+\frac35\right).\)

At the point where it meets \(y=x\), set \(y=x\):

\(x-3=-\frac15\left(x+\frac35\right).\)

Multiplying by \(5\),

\(5x-15=-x-\frac35.\)

So

\(6x=\frac{72}{5},\)

and hence

\(x=\frac{12}{5}.\)

Since \(y=x\),

\(y=\frac{12}{5}.\)

The required point is

\(\left(\frac{12}{5},\frac{12}{5}\right).\)