Let \(G\) be generated by \(a,b\) subject to \[ a^{-1}b^2a=b^3, \qquad b^{-1}a^2b=a^3. \] Is \(G\) trivial?
Solution
1. For every integer \(m\),\[ b^{-1}a^{2m}b=a^{3m}. \]
The second defining relation gives \[ b^{-1}a^2b=a^3. \] Conjugation is an automorphism, so \[ b^{-1}a^{2m}b = \left(b^{-1}a^2b\right)^m = a^{3m}. \]
2. One has\[ b^{-3}a^8b^3=a^{27}. \]
2.1. One has\[ b^{-1}a^8b=a^{12}. \]
Apply step 1 with \(m=4\).
2.2. One has\[ b^{-2}a^8b^2=a^{18}. \]
By step 1, \[ b^{-2}a^8b^2 = b^{-1}a^{12}b. \] Apply step 1 with \(m=6\).
2.3. One has\[ b^{-3}a^8b^3=a^{27}. \]
By step 2, \[ b^{-3}a^8b^3 = b^{-1}a^{18}b. \] Apply step 1 with \(m=9\).
2.4. Q.E.D.
Step 3 is the claim of step 2.
3. The first defining relation implies\[ ab^3=b^2a \qquad\text{and}\qquad b^{-3}a^{-1}=a^{-1}b^{-2}. \]
Multiplying \[ a^{-1}b^2a=b^3 \] on the left by \(a\) gives \[ b^2a=ab^3. \] Taking inverses gives \[ b^{-3}a^{-1}=a^{-1}b^{-2}. \]
4. One has\[ a^{27}=b^{-2}a^8b^2. \]
3. Multiplying on the left by \(a\)and on the right by \(a^{-1}\) yields\[ a^{27}=b^{-2}a^8b^2. \]
5. One has\[ a^9=1. \]
Step 2 gives \[ b^{-2}a^8b^2=a^{18}. \] Together with step 4 this gives \[ a^{27}=a^{18}. \] Cancelling \(a^{18}\) yields \(a^9=1\).
6. One has\[ a^4=1. \]
By step 1 with \(m=2\), \[ b^{-1}a^4b=a^6. \] Therefore \[ \begin{aligned} b^{-2}a^4b^2 &=b^{-1}a^6b\\ &=a^9, \end{aligned} \] where the last equality is step 1 with \(m=3\). Step 5 gives \(a^9=1\), so \[ b^{-2}a^4b^2=1. \] Conjugating by \(b^2\) gives \(a^4=1\).
7. One has\[ a=1. \]
Steps 5 and 6 give \(a^9=a^4=1\). Since \[ 1=9-2\cdot4, \] we have \[ a=a^9(a^4)^{-2}=1. \]
8. One has\[ b=1. \]
Substituting step 7 into \[ a^{-1}b^2a=b^3 \] gives \(b^2=b^3\). Cancelling \(b^2\) yields \(b=1\).
9. Therefore\[ \boxed{G=\{1\}}. \]
By steps 7 and 8, both generators \(a\) and \(b\) are the identity. Hence the group they generate is trivial.
10. Q.E.D.
Step 9 answers the question.