Let \(T:V\to V\) be a linear operator on an \(n\)-dimensional vector space over a field \(F\). Prove that \(T\) has an invariant subspace \(W\) with \[ \{0\}\ne W\ne V \] if and only if the characteristic polynomial of \(T\) has a factor \(f\in F[t]\) satisfying \[ 0<\deg f<n. \]
Solution
1. If \(T\) has a nonzero proper invariant subspace \(W\), then itscharacteristic polynomial has a factor of degree strictly between \(0\) and\(n\).
Let \(d=\dim W\), so \(0<d<n\). Choose a basis of \(W\) and extend it to a basis of \(V\). Since \(W\) is \(T\)-invariant, the matrix of \(T\) in this basis has block upper-triangular form \[ [T]= \begin{pmatrix} A&C\\ 0&B \end{pmatrix}, \] where \(A\) is \(d\times d\). Hence \[ \chi_T(t) = \det(tI-[T]) = \det(tI_d-A)\det(tI_{n-d}-B). \] Thus \(f(t)=\det(tI_d-A)\) is a factor of \(\chi_T\) with \(\deg f=d\), and \(0<d<n\).
2. If \(T\) has no nonzero proper invariant subspace, then \(V\) is a simple\(F[t]\)-module under the action\[ p(t)\cdot v=p(T)v. \]
The \(F[t]\)-submodules for this action are exactly the \(F\)-linear subspaces stable under \(T\). Thus the assumption says precisely that the only \(F[t]\)-submodules are \(0\) and \(V\).
3. Under the hypothesis of step 2, the characteristic polynomial\(\chi_T\) is irreducible over \(F\).
Choose \(0\neq v\in V\). Since \(V\) is simple, the nonzero submodule \(F[t]v\) equals \(V\). Therefore \[ \phi:F[t]\longrightarrow V, \qquad p\longmapsto p(T)v, \] is surjective. Its kernel is an ideal \((q)\), and \[ V\cong F[t]/(q) \] as \(F[t]\)-modules. Since \(V\) is simple, \((q)\) is maximal. The maximal ideals of the polynomial ring \(F[t]\) are generated by irreducible polynomials, so \(q\) may be chosen monic and irreducible.
The vectors \[ v,Tv,\ldots,T^{d-1}v, \qquad d=\deg q, \] correspond to the standard basis of \(F[t]/(q)\) and hence form a basis of \(V\). In this basis, \(T\) is multiplication by \(t\) modulo \(q\), whose companion matrix has characteristic polynomial \(q\). Consequently \[ \chi_T=q, \] which is irreducible.
4. If \(\chi_T\) has a factor \(f\in F[t]\) with\(0<\deg f<n\), then \(T\) has a nonzero proper invariant subspace.
Such a factorization makes the degree-\(n\) polynomial \(\chi_T\) reducible. The contrapositive of step 3 says that \(T\) cannot have no nonzero proper invariant subspace. Hence such a subspace exists.
5. Q.E.D.
Step 1 proves one implication and step 4 proves the converse.