An \(n\times n\) real matrix \(T\) is called positive definite if it is symmetric and \[ \langle Tx,x\rangle>0 \] for every nonzero \(x\in\mathbb R^n\). Let \(A\) and \(B\) be positive-definite real matrices.
Show that there is a basis \(\{v_1,\ldots,v_n\}\) of \(\mathbb R^n\) and real numbers \(\lambda_1,\ldots,\lambda_n\) such that \[ \langle Av_i,v_j\rangle= \begin{cases} 1,&i=j,\\ 0,&i\ne j, \end{cases} \] and \[ \langle Bv_i,v_j\rangle= \begin{cases} \lambda_i,&i=j,\\ 0,&i\ne j. \end{cases} \]
Deduce that there is an invertible real matrix \(U\) such that \[ U^TAU=I \qquad\text{and}\qquad U^TBU \] is diagonal.
Solution
1. The formula\[ (x,y)_A\coloneqq\langle Ax,y\rangle \]defines an inner product on \(\RR^n\).
Because \(A\) is symmetric, \[ (x,y)_A = \langle Ax,y\rangle = \langle x,Ay\rangle = (y,x)_A. \] Bilinearity is immediate, and positive definiteness of \(A\) gives \[ (x,x)_A=\langle Ax,x\rangle>0 \] for every nonzero \(x\).
2. The matrix \(A\) is invertible, and the operator\[ T\coloneqq A^{-1}B \]is self-adjoint with respect to \((\ ,\ )_A\).
If \(Ax=0\), then \[ \langle Ax,x\rangle=0, \] so positive definiteness forces \(x=0\). Thus \(A\) has trivial kernel and is invertible.
For \(x,y\in\RR^n\), \[ \begin{aligned} (Tx,y)_A &= \langle A(A^{-1}B)x,y\rangle\\ &= \langle Bx,y\rangle\\ &= \langle x,By\rangle\\ &= \langle Ax,A^{-1}By\rangle\\ &= (x,Ty)_A, \end{aligned} \] where symmetry of \(B\) is used in the third equality and symmetry of \(A\) in the fourth.
3. There is an \((\ ,\ )_A\)-orthonormal basis\[ v_1,\ldots,v_n \]of \(\RR^n\) consisting of eigenvectors of \(T\):\[ Tv_i=\lambda_i v_i \]for real numbers \(\lambda_i\).
By step 2, \(T\) is self-adjoint on the finite-dimensional real inner-product space \((\RR^n,(\ ,\ )_A)\). The spectral theorem for self-adjoint operators therefore gives an orthonormal eigenbasis with real eigenvalues.
4. The basis from step 3 satisfies\[ \langle Av_i,v_j\rangle=\delta_{ij} \]and\[ \langle Bv_i,v_j\rangle = \lambda_i\delta_{ij}. \]
The first identity is exactly the \((\ ,\ )_A\)-orthonormality of the basis. Since \[ B=AT, \] one has \[ \begin{aligned} \langle Bv_i,v_j\rangle &= \langle ATv_i,v_j\rangle\\ &= \lambda_i\langle Av_i,v_j\rangle\\ &= \lambda_i\delta_{ij}. \end{aligned} \] This proves part 1. In fact, positive definiteness of \(B\) also gives \(\lambda_i>0\), since \[ \lambda_i = \langle Bv_i,v_i\rangle. \]
5. Let \(U\) be the matrix whose \(i\)th column is \(v_i\). Then\[ U^TAU=I \]and\[ U^TBU=\operatorname{diag}(\lambda_1,\ldots,\lambda_n). \]
Because \(v_1,\ldots,v_n\) is a basis, \(U\) is invertible. The \((i,j)\) entry of \(U^TAU\) is \[ v_i^TAv_j = \langle Av_j,v_i\rangle = \delta_{ij}, \] so \(U^TAU=I\). Likewise, the \((i,j)\) entry of \(U^TBU\) is \[ v_i^TBv_j = \langle Bv_j,v_i\rangle = \lambda_j\delta_{ij}, \] which is the stated diagonal matrix.
6. Q.E.D.
Step 4 proves part 1, and step 5 proves part 2.