Let \(P\) be a nonconstant complex polynomial. Show that every zero of \(P'\) lies in the convex hull of the zeros of \(P\).
The nonconstant hypothesis is necessary. For \(P=1\), the derivative is identically zero and vanishes at every complex point, whereas \(P\) has no roots and their convex hull is empty. Thus the containment fails in that case.
Solution
Let \(n=\deg P\geq1\). The fundamental theorem of algebra gives roots \(\zeta_1,\ldots,\zeta_n\), repeated with their multiplicities [@SS03]. Their convex hull is \[ K=\left\{\sum_{j=1}^n t_j\zeta_j:t_j\geq0,\ \sum_{j=1}^n t_j=1\right\}. \] It is nonempty and compact: it is the continuous image of the nonempty closed, bounded simplex of the coefficient vectors \((t_1,\ldots,t_n)\) in \(\mathbb R^n\).
1. A critical point that is itself a zero of \(P\) already lies in \(K\).
If \(w\) satisfies \(P'(w)=0\) and also \(P(w)=0\), then \(w\) is one of the zeros used to form the convex hull \(K\). Hence \(w\in K\). It remains only to consider a critical point \(w\) with \(P(w)\ne0\).
2. A point outside \(K\) admits a direction in which every vector from a root to that point has positive real projection.
Suppose \(w\notin K\). The set \(K\) is compact and convex, so there is a point \(q\in K\) minimizing \(|w-q|\). Put \(v=w-q\ne0\).
For any \(\zeta\in K\) and \(0\le t\le1\), convexity gives \(q+t(\zeta-q)\in K\). Minimality of \(q\) implies \[ |w-q|^2\le |w-q-t(\zeta-q)|^2. \] Expanding and dividing by \(t>0\), then letting \(t\downarrow0\), gives \[ \operatorname{Re}\bigl(\overline v(\zeta-q)\bigr)\le0. \] Therefore \[ \operatorname{Re}\bigl(v\,\overline{(w-\zeta)}\bigr) =|v|^2-\operatorname{Re}\bigl(\overline v(\zeta-q)\bigr) \ge |v|^2>0. \] In particular this holds for every zero \(\zeta\) of \(P\).
3. The logarithmic derivative rules out a critical point outside \(K\).
Factor \[ P(z)=c\prod_{j=1}^n(z-\zeta_j), \] where the roots are repeated according to multiplicity. At a point \(w\) with \(P(w)\ne0\), \[ \frac{P'(w)}{P(w)}=\sum_{j=1}^n\frac1{w-\zeta_j}. \] If in addition \(w\notin K\), step 2 gives for every \(j\) \[ \operatorname{Re}\left(\frac{v}{w-\zeta_j}\right) =\frac{\operatorname{Re}\bigl(v\overline{(w-\zeta_j)}\bigr)}{|w-\zeta_j|^2}>0. \] Hence \[ \operatorname{Re}\left( v\frac{P'(w)}{P(w)}\right) =\sum_{j=1}^n \operatorname{Re}\left(\frac{v}{w-\zeta_j}\right)>0. \] This is impossible if \(P'(w)=0\). Thus no zero of \(P'\) lying outside the root set of \(P\) can be outside \(K\). Together with step 1, every zero of \(P'\) lies in the convex hull of the zeros of \(P\).