Let \(F\) be a commuting family of \(3 \times 3\) complex matrices. How many linearly independent matrices can \(F\) contain? What about the \(n \times n\) case?
Solution
The maximum number is \[ \boxed{\left\lfloor\frac{n^2}{4}\right\rfloor+1.} \] For \(n=3\) this is \(\boxed{3}\).
Let \[ d_n=\left\lfloor\frac{n^2}{4}\right\rfloor+1. \]
1. It suffices to bound the dimension of a commuting vector subspace of \(M_n(\mathbb C)\).
If \(\mathcal F\) is a pairwise commuting family, then its complex linear span \[ \mathcal A=\operatorname{span}_{\mathbb C}(\mathcal F) \] is again pairwise commuting, because bilinearity gives \[ \left[\sum_i a_iA_i,\sum_j b_jA_j\right] =\sum_{i,j}a_ib_j[A_i,A_j]=0. \] Thus the number of linearly independent members of \(\mathcal F\) is at most \(\dim\mathcal A\). Conversely, a basis of any commuting vector subspace is a commuting linearly independent family. Hence the problem is exactly to find the largest possible dimension of a commuting vector subspace of \(M_n(\mathbb C)\).
2. Every commuting vector subspace of \(M_n(\mathbb C)\) has dimension at most \(d_n\).
We argue by induction on \(n\). For \(n=1\) the assertion is immediate, since \(M_1(\mathbb C)\) has dimension \(1=d_1\).
Assume the result for size \(n-1\), and let \(\mathcal A\subseteq M_n(\mathbb C)\) be a commuting vector subspace. By simultaneous triangularization of a commuting family over the algebraically closed field \(\mathbb C\), one change of basis makes every element of \(\mathcal A\) upper triangular [@HK71]. Conjugation is a vector-space isomorphism and preserves commutation, so we may assume from the outset that every element of \(\mathcal A\) is upper triangular.
Let \[ \pi:\mathcal A\longrightarrow M_{n-1}(\mathbb C) \] send a matrix to its lower-right \((n-1)\times(n-1)\) principal block. The image is a commuting vector subspace, so by induction \[ k:=\dim\pi(\mathcal A)\le d_{n-1}. \] If \(d=\dim\mathcal A\), rank-nullity gives \[ \dim\ker\pi=d-k. \] Because matrices in \(\mathcal A\) are upper triangular, a matrix in \(\ker\pi\) can have nonzero entries only in its first row. Choose a basis \(B_1,\ldots,B_{d-k}\) of \(\ker\pi\), and write its first rows as \(r_1,\ldots,r_{d-k}\in\mathbb C^n\). These row vectors are linearly independent.
Similarly, let \[ \rho:\mathcal A\longrightarrow M_{n-1}(\mathbb C) \] take the upper-left \((n-1)\times(n-1)\) principal block. Its image is commuting, so \[ s:=\dim\rho(\mathcal A)\le d_{n-1}, \qquad \dim\ker\rho=d-s. \] An upper-triangular matrix in \(\ker\rho\) can have nonzero entries only in its last column. Choose a basis \(C_1,\ldots,C_{d-s}\) of this kernel, with last columns \(c_1,\ldots,c_{d-s}\in\mathbb C^n\). These column vectors are linearly independent.
Writing \(e_1,e_n\) for the first and last standard basis columns, we have \[ B_i=e_1r_i, \qquad C_j=c_je_n^{T}. \] Since \(n>1\), \[ C_jB_i=c_j(e_n^Te_1)r_i=0, \] whereas \[ B_iC_j=(r_ic_j)e_1e_n^T. \] All elements of \(\mathcal A\) commute, so \(B_iC_j=C_jB_i\) and therefore \[ r_ic_j=0 \] for every \(i,j\).
Let \(R\) be the \((d-k)\times n\) matrix whose rows are the \(r_i\). Their linear independence gives \[ \operatorname{rank}R=d-k. \] Every \(c_j\) lies in \(\ker R\), and the \(c_j\) are linearly independent, hence \[ \operatorname{nullity}R\ge d-s. \] Rank-nullity in \(\mathbb C^n\) now yields \[ n\ge(d-k)+(d-s) \ge2(d-d_{n-1}). \] Thus \[ d\le d_{n-1}+\frac n2. \] To get the exact integer bound, suppose instead that \(d\ge d_n+1\). Then \[ d-d_{n-1} \ge \left\lfloor\frac{n^2}{4}\right\rfloor -\left\lfloor\frac{(n-1)^2}{4}\right\rfloor+1. \] If \(n=2m\), the right side is \(m+1\); if \(n=2m+1\), it is again \(m+1\). Consequently \[ 2(d-d_{n-1})>n, \] contradicting the rank-nullity inequality above. Therefore \(d\le d_n\).
3. The bound \(d_n\) is attained.
Put \[ p=\left\lfloor\frac n2\right\rfloor, \qquad q=n-p=\left\lceil\frac n2\right\rceil. \] For \(a\in\mathbb C\) and arbitrary \(B\in M_{p\times q}(\mathbb C)\), consider \[ X(a,B)= \begin{pmatrix} aI_p&B\\ 0&aI_q \end{pmatrix}. \] For two such matrices, \[ X(a,B)X(b,C) = \begin{pmatrix} abI_p&aC+bB\\ 0&abI_q \end{pmatrix} =X(b,C)X(a,B). \] Hence these matrices form a commuting vector subspace. The scalar \(a\) gives one dimension and \(B\) contributes \(pq\) dimensions, so its dimension is \[ 1+pq =1+\left\lfloor\frac{n^2}{4}\right\rfloor =d_n. \] Thus the upper bound from step 2 is sharp. For \(n=3\), it gives \(d_3=\lfloor9/4\rfloor+1=3\).