Integrate by parts twice: \[\begin{aligned}\int x^2\cos(x/3)dx&=3x^2\sin(x/3)-6\int x\sin(x/3)dx\\&=3x^2\sin(x/3)+18x\cos(x/3)-54\sin(x/3).\end{aligned}\] At 0 this is 0. At \(\pi\), \(\sin(\pi/3)=\sqrt3/2\), \(\cos(\pi/3)=1/2\), so \[I=\frac{3\sqrt3\pi^2}2+9\pi-27\sqrt3.\]