Characterization of minimal counterexamples to a characteristic p-functor controlling normal p-complements

Statement

Suppose $p$ is a prime number and $W$ is a characteristic p-functor that does not control normal p-complements. Suppose $G$ is a finite group of the least possible order where such a phenomenon is realized: $G$ has a nontrivial $p$-Sylow subgroup $P$ such that $N_{G}(W(P))$ has a normal p-complement (i.e., is a p-nilpotent group) but $G$ does not (i.e., $G$ is not a p-nilpotent group). Then, the following are true:

1. $\!O_{p'}(G)$ is trivial: In other words, $G$ does not possess any nontrivial normal subgroup of order relatively prime to $p$.
2. $\!O_{p}(G)$ is nontrivial but not equal to $P$.
3. $\!G/O_{p}(G)$ possesses a normal $p$-complement. In particular, $\!G=O_{p,p',p}(G)$.

Facts used

1. Frobenius' normal p-complement theorem
2. Characteristic of normal implies normal

Proof

Proof of (1)

Given: A finite group $G$ that is of smallest order with respect to the property that there exists a nontrivial $p$-Sylow subgroup $P$ such that $N_{G}(W(P))$ possesses a normal $p$-complement but $G$ does not.

To prove: $\!O_{p'}(G)$ is trivial.

Proof strategy: Suppose $O_{p'}(G)$ were nontrivial. We will then derive a contradiction from the fact that $G/O_{p'}(G)$, being of smaller order, is not a counterexample.

Proof details: Consider the quotient map $\varphi :G\to G/O_{p'}(G)$.

Step no.	Assertion/construction	Given data used	Facts used	Previous steps used	Explanation
1	$\varphi$ restricts to an isomorphism on $p$-subgroups. In particular, the restriction of $\varphi$ to $P$ gives an isomorphism from $P$ to $\varphi (P)$				The kernel of $\varphi$ is $O_{p'}(G)$, which has order relatively prime to $p$.
2	$\varphi (P)$ is a $p$-Sylow subgroup of $\varphi (G)$
3	$\varphi (W(P))=W(\varphi (P))$.		$W$ is a characteristic $p$-functor	Step (1)	Follows from step (1) and the fact that $W$ is a characteristic $p$-functor.
4	$\varphi (N_{G}(W(P)))$ is $p$-nilpotent, i.e., it has a normal $p$-complement.	Fact (1)	$N_{G}(W(P))$ has a $p$-complement, i.e., is $p$-nilpotent
5	$\varphi (N_{G}(W(P)))=N_{\varphi (G)}(W(\varphi (P))$.			Step (3)	[SHOW MORE] By Step (3), we already have $\varphi (W(P))=W(\varphi (P))$. This gives $N_{\varphi (G)}(\varphi (W(P)))=N_{\varphi (G)}(W(\varphi (P))$. Surjective homomorphisms commute with taking normalizers, so the left side can be rewritten as Failed to parse (unknown function "\varph"): {\displaystyle \varph(N_G(W(P)))} , giving the result.
6	$N_{\varphi (G)}(W(\varphi (P))$ is $p$-nilpotent, i.e., possesses a normal $p$-complement.			Steps (4), (5)	Step-combination direct
7	If $O_{p'}(G)$ is nontrivial, then $\varphi (G)$ is $p$-nilpotent, i.e., possesses a normal $p$-complement.		$G$ is a minimal order counterexample	Steps (2), (6)	<toggledisplay>If $O_{p'}(G)$ is nontrivial, then $\varphi (G)=G/O_{p'}(G)$ is of strictly smaller order than $G$. By the minimal counterexample assumption, $W$ controls normal $p$-complements in $\varphi (G)$. Combining with steps (2) and (6), we obtain that $\varphi (G)$ is $p$-nilpotent.
8	If $O_{p'}(G)$ is nontrivial, then $G$ is $p$-nilpotent.			Step (7)	By Step (7), $\varphi (G)=G/O_{p'}(G)$ is $p$-nilpotent, hence so is $G$. Specifically, the normal $p$-complement in $G$ is the full inverse image under $\varphi$ of the normal $p$-complement in $\varphi (G)$.

Proof of (2)

Given: A finite group $G$ that is of smallest order with respect to the property that there exists a nontrivial $p$-Sylow subgroup $P$ such that $N_{G}(W(P))$ possesses a normal $p$-complement but $G$ does not.

To prove: $O_{p}(G)$ is a proper nontrivial subgroup of $P$.

Proof:

1. By fact (1), there exists a non-identity $p$-subgroup $Q$ of $G$ such that $N=N_{G}(Q)$ does not have a normal $p$-complement. Among such $Q$, pick a $Q$ such that the order of the Sylow $p$-subgroup of $N_{G}(Q)$ is maximal. By conjugation, we can further assume that $P$ contains the Sylow subgroup of $N_{G}(Q)$, so this Sylow subgroup is $P\cap N$.
2. $P\leq N$: Suppose not. Then set $R=P\cap N$, and set $L=N_{N}(W(R))),M=N_{G}(W(R)))$.
   • $P\cap M$ contains $N_{P}(R)$: Since $W(R)$ is a characteristic subgroup of $R$, $W(R)$ is normal in $N_{P}(R)$ (by fact (2)). Thus, $N_{P}(R)\leq M$. In particular, $P\cap M$ contains $N_{P}(R)$.
   • $P\cap M$ properly contains $R$: Since $R$ is proper in $P$, $R$ is proper in $N_{P}(R)$. Thus, $R$ is properly contained in $P\cap M$ by the previous step.
   • $M$ has a normal $p$-complement: We have found a subgroup $R$ such that $R\leq P$ and the $p$-Sylow subgroup of $N_{G}(R)$ is strictly bigger in size that that of $N_{G}(Q)$. By the maximality assumption, $M=N_{G}(R)$ must have a normal $p$-complement.
   • $L$ has a normal $p$-complement: This follows from the previous statement, and the observation that $L\leq M$.
   • $N$ has a normal $p$-complement: If $N=G$, then $P\leq N$, which we're assuming is false. Thus, $N$ is proper in $G$. By the previous step, $N$ satisfies the conditions for the induction hypothesis, so $N$ has a normal $p$-complement.
   • The desired contradiction is achieved: The desired contradiction is achieved because by our assumption, $N=N_{G}(Q)$ does not possess a normal $p$-complement.
3. $N=G$ and $O_{p}(G)$ is nontrivial. Without loss of generality, $Q=O_{p}(G)$: If $N$ were a proper subgroup of $G$, we'd have $P\leq N\leq G$ and it would possess a normal $p$-complement by step (2). This contradicts the assumption that $N=N_{G}(Q)$ does not have a normal $p$-complement. Further, since $N_{G}(Q)=G$ and $Q$ is nontrivial, $Q$ is a nontrivial normal $p$-subgroup. Thus, $O_{p}(G)$ is nontrivial, and we can assume without loss of generality that $Q=O_{p}(G)$.
4. If $O_{p}(G)=P$, it has a normal $p$-complement. Thus, we may assume that $Q=O_{p}(G)$ is a proper subgroup of $P$: By fact (2) again, $W(P)$ is normal in $G$. Thus, $G=N_{G}(W(P))$ and the assumption that $N_{G}(W(P))$ has a normal $p$-complement yields that $G$ has a normal $p$-complement.

Proof of (3)

Given: $G$ is a group of smallest order with respect to the following: $G$ has a nontrivial $p$-Sylow subgroup such that $N_{G}(W(P))$ has a normal $p$-complement but $G$ does not.

To prove: $G=O_{p,p',p}(G)$.

Proof: We've already established in part (2) that $Q=O_{p}(G)$ is a proper nontrivial subgroup of $P$. We proceed from there.

Let $\varphi :G\to G/Q$ be the quotient map. Then $\varphi (P)=P/Q$ is $p$-Sylow in $\varphi (G)$. Let $M_{1}=\varphi ^{-1}(W(P/Q))$ and $N_{1}=\varphi ^{-1}(N_{G/Q}(W(P/Q))))$.

1. $N_{1}=N_{G}(M_{1})$: This follows from some form of the lattice isomorphism theorem, and the fact that both subgroups contain the kernel of $\varphi$.
2. If $P\leq T\leq G$, and $T$ is proper in $G$, $P$ has a normal complement in $T$:
   • If $P\leq T\leq G$, then $P$ is a $p$-Sylow subgroup of $T$, because Sylow satisfies intermediate subgroup condition. Further, $N_{T}(W(P))$ is a subgroup of $N_{G}(W(P))$ containing $P$. In particular, since $P$ has a normal complement in $N_{G}(W(P))$, it has a normal complement in $N_{T}(W(P))$, because retract satisfies intermediate subgroup condition.
   • In particular, if we assume the induction hypothesis holds for $T$, then we deduce that $P$ has a normal complement in $T$.
3. $N_{1}$ has a normal $p$-complement: This is a direct consequence of the previous step, and the observation that since $\varphi (P)\leq \varphi (N_{1})$, $P\leq N_{1}$.
4. $\varphi (N_{1})$ has a normal $p$-complement: This follows from the preceding step.
5. $\varphi (G)$ has a normal $p$-complement: From the previous step, we see that $N_{G/Q}(W(P/Q))$ has a normal $p$-complement. $\varphi (G)=G/Q$, having order strictly smaller than that of $G$, satisfies the property that $W$ controls normal $p$-complements in it. Thus, $\varphi (G)$ has a normal $p$-complement.
6. $G=O_{p,p',p}(G)$: From the previous step, $\varphi (G)=O_{p',p}(\varphi (G))$, yielding that $G=O_{p,p',p}(G)$.