Let \(f:\Delta\to\Delta\) be holomorphic with \(f(0)=0\). Prove that either \(f\) is a rotation, or \[ f^n(z)\longrightarrow0 \qquad(n\to\infty), \] where \(f^n=f\circ f\circ\cdots\circ f\) denotes the \(n\)-fold iterate.
Solution
1. By Schwarz’s lemma,\[ |f(z)|\le |z| \qquad(z\in\Delta). \]Moreover, if equality holds at any nonzero point, then \(f\) is a rotation.
The hypotheses of Schwarz’s lemma apply because \(f:\Delta\to\Delta\) is holomorphic and \(f(0)=0\). Its equality case says that if \(|f(z_0)|=|z_0|\) for some \(z_0\neq0\), then \[ f(z)=e^{i\theta}z \] for some real \(\theta\).
2. Assume \(f\) is not a rotation. Then\[ g(z)= \begin{cases} f(z)/z,&z\neq0,\\ f'(0),&z=0 \end{cases} \]is holomorphic on \(\Delta\) and satisfies\[ |g(z)|<1 \qquad(z\in\Delta). \]
Since \(f(0)=0\), the quotient \(f(z)/z\) has a removable singularity at \(0\), whose value after removal is \(f'(0)\). Thus \(g\) is holomorphic. By 1, \(|g(z)|\le1\). If equality held at any point, then the equality case of Schwarz’s lemma would make \(f\) a rotation, contrary to assumption. Hence the inequality is strict everywhere.
3. Fix \(z\in\Delta\) and set \(r=|z|\). If \(r>0\), there is a constant\(q=q(r)<1\) such that\[ |f(w)|\le q|w| \qquad(|w|\le r). \]
The continuous function \(|g|\) attains a maximum on the compact disk \(\{|w|\le r\}\). By 2 this maximum is strictly less than \(1\); call it \(q\). Since \(f(w)=wg(w)\), the stated estimate follows.
4. For every \(z\in\Delta\),\[ f^n(z)\longrightarrow0. \]
The case \(z=0\) is immediate because every iterate fixes \(0\). Let \(z\neq0\) and put \(r=|z|\). From 1, \[ |f^k(z)|\le r \] for every \(k\ge0\). Therefore the estimate in 3 applies at every step of the orbit, and induction gives \[ |f^n(z)|\le q^n|z|. \] Since \(0\le q<1\), the right-hand side tends to zero. Hence \(f^n(z)\to0\).
5. Thus exactly the required dichotomy holds: either\(f(z)=e^{i\theta}z\) is a rotation, or all iterates converge pointwise to\(0\) on \(\Delta\).
If \(f\) is a rotation, this is the first alternative. Otherwise 4 proves the second.