Let \(V\) be the space of all \(n \times 1\) matrices over \(C\), \(A\) an \(n \times n\) matrix over \(C\), and \(f\) the bilinear form on \(V\) given by \(f(X, Y) = X^tAY\). Show that \(f\) is invariant under \(O(n, C)\), i.e., \(f(MX, MY) = f(X, Y)\) for all \(X, Y\) in \(V\) and \(M\) in \(O(n, C)\), if and only if \(A\) commutes with each member of \(O(n, C)\).
Solution
For \(M\in O(n,\mathbb C)\) and \(X,Y\in V\), \[ \begin{aligned} f(MX,MY) &=(MX)^tA(MY)\\ &=X^tM^tAMY. \end{aligned} \] Thus \[ f(MX,MY)=f(X,Y) \] for all \(X,Y\) if and only if \[ M^tAM=A. \] Indeed, equality of the two bilinear forms for all \(X,Y\) forces equality of their representing matrices.
Since \(M\in O(n,\mathbb C)\), \[ M^t=M^{-1}. \] Therefore \[ M^tAM=A \iff M^{-1}AM=A \iff AM=MA. \] This equivalence holds for every \(M\in O(n,\mathbb C)\). Hence \[ \boxed{ f\text{ is }O(n,\mathbb C)\text{-invariant} \iff AM=MA\text{ for every }M\in O(n,\mathbb C).} \]