Answer:
(a)
\(\bar{x}=\frac13(3ka+h), \qquad \bar{y}=\frac13(2a+h).\)
(b)
\(0\le h\le \sqrt3ka.\)
(c)
\((\bar{x},\bar{y})=\left(\frac{a(\sqrt3+1)}{3},a\right).\)
(a) From the diagram, take coordinates relative to \(O\). The triangle \(ABC\) has vertices
\(A(ka,0), \qquad B(ka+h,h), \qquad C(ka,2a).\)
The centre of mass of a uniform triangular lamina is the average of the coordinates of its vertices. Hence
\(\bar{x}=\frac{ka+(ka+h)+ka}{3} =\frac{3ka+h}{3} =\frac13(3ka+h).\)
Similarly,
\(\bar{y}=\frac{0+h+2a}{3} =\frac13(2a+h).\)
(b) Now find the \(x\)-coordinate of the centre of mass of the whole lamina \(OABCD\).
The rectangle \(OACD\) has area \(2ka^2\) and centre of mass at \(x=\frac12ka\). The triangle \(ABC\) has area
\(\frac12(2a)h=ah\)
and its centre of mass has \(x\)-coordinate \(ka+\frac13h\). Let the \(x\)-coordinate of the centre of mass of the whole lamina be \(X\). Taking moments about the \(y\)-axis through \(O\),
\((2ka^2+ah)X =2ka^2\left(\frac12ka\right)+ah\left(ka+\frac13h\right).\)
Thus
\(X=\frac{k^2a^3+ka^2h+\frac13ah^2}{2ka^2+ah} =\frac{k^2a^2+kah+\frac13h^2}{2ka+h}.\)
When the lamina rests on the edge \(OA\), it remains in equilibrium provided the vertical line through its centre of mass falls within the base \(OA\). Hence
\(X\le ka.\)
Substitute the expression for \(X\):
\(\frac{k^2a^2+kah+\frac13h^2}{2ka+h}\le ka.\)
Since \(2ka+h\gt 0\),
\(k^2a^2+kah+\frac13h^2\le 2k^2a^2+kah.\)
So
\(\frac13h^2\le k^2a^2,\)
and therefore
\(0\le h\le \sqrt3ka.\)
(c) Here
\(k=\frac{\sqrt3}{3}.\)
On the point of toppling,
\(h=\sqrt3ka =\sqrt3\left(\frac{\sqrt3}{3}\right)a =a.\)
For the triangle \(ABC\),
\(\bar{x}=ka+\frac13h =\frac{\sqrt3}{3}a+\frac13a =\frac{a(\sqrt3+1)}{3},\)
and
\(\bar{y}=\frac13(2a+h)=\frac13(2a+a)=a.\)
Hence the centre of mass of triangle \(ABC\) is
\(\left(\frac{a(\sqrt3+1)}{3},a\right).\)