INDICES OF MAXIMAL INVARIANT SUBGROUPS AND SOLVABILITY OF FINITE GROUPS

Let A and G be finite groups and suppose that A acts coprimely on G via automorphisms. We study the solvability and supersolvability of G when certain proper maximal A-invariant subgroups of G have prime index or when they have certain prime power indices in G.


Introduction
Maximal subgroups play an important role in researching the solvability and supersolvability of finite groups. This is a classic theme in finite group theory that is in continuous developement (see for instance [9] and [10]). Among the known results, an almost straightforward result asserts that when all maximal subgroups of a group G have prime index then G is solvable. A later theorem of B. Huppert claims that such property actually characterizes supersolvable groups [6, VI.9.2 and VI. 9.5]. This is also a particular case of a theorem of P. Hall, which establishes the solvability of G when the indices of the maximal subgroups are just primes or squares of primes ([6, VI.9.4]). However, seeking solvability one cannot go further in this direction since, for instance, the maximal subgroups of the simple group PSL 2 (7) have exactly index 7 and 8. More precisely, R. Guralnick showed in [5], by means of the Classification of the Finite Simple Groups, that PSL 2 (7) is the unique simple group in which every maximal subgroup has prime power index.
Suppose now that G and A are finite groups of relative coprime orders such that G is acted on via automorphisms by the group A. Under this coprime action scenario, it is not difficult to prove that when G is solvable, then every maximal A-invariant subgroup of G must have prime power index (see for instance [2,Lemma 2.3]). Now, what about the converse assertion? Of course, Guralnick's paper shows that this is not true just by considering the trivial action. In this note, we investigate what information on the indices of the maximal A-invariant subgroups of G may provide solvability or supersolvability. Our first sufficient condition extends Hall's criterion in a natural way and is the following.
Theorem A. Let G and A be finite groups of coprime orders and assume that A acts on G by automorphisms. If the index of every maximal A-invariant subgroup of G is prime or the square of a prime, then G is solvable.
It is easy to see that the condition that every maximal A-invariant subgroup of a group has prime index is not necessarily satisfied by supersolvable groups, contrary to what happens in the ordinary situation. For instance, the (supersolvable) quaternion group of order 8, say Q, has an automorphism of order 3, say α, in such a way that Q possesses exactly one α -invariant proper subgroup, which is obviously maximal invariant of index 4. However, with respect to the converse direction, that is, whether the condition that the index of every maximal A-invariant subgroup is a prime should imply supersolvability, we give a positive answer.
Theorem B. Let G and A be finite groups of coprime orders and assume that A acts on G by automorphisms. If the index of every maximal A-invariant subgroup of G is prime, then G is supersolvable.
Regarding solvability, it should be noted that the authors have proved in [3] that if the index of every non-nilpotent maximal A-invariant subgroup of G is a prime, then G is solvable. In this note, we can take one step more. In fact, the particular case of Theorem A in which all indices are prime numbers is an immediate consequence of our main result, in which there is no need to impose every maximal invariant subgroup to have prime index so as to obtain solvability.
Theorem C. Let G and A be finite groups of coprime orders and assume that A acts on G by automorphisms. If every proper non-maximal A-invariant subgroup of G lies in some maximal A-invariant subgroup of G that has prime index in G, then G is solvable.
While the proofs of Theorems A and B are elementary, the approach of the proof of Theorem C consists in getting a reduction to certain almost simple groups (by using a Guralnick's classification theorem, Lemma 2.6) in order to be conducted later a coprime action analysis on these almost simple groups. All groups are supposed to be finite and we will use the standard notation appearing in [8].

Preliminaries
In this section, we present most of the results that will be needed for our purposes. The first one is required in the proof of Theorem C for the case in which the action of A on G is trivial. We use F (G) to denote the Fitting subgroup of G.
Lemma 2.1. [11,Theorem] Given a finite group G whose every proper non-maximal subgroup lies in some subgroup of prime index, the quotient G/F (G) is supersolvable. Theorem 8.13] Let G be a finite group. Suppose that p ≥ 5, P ∈ Syl p (G) and P = 1. If N G (P )/C G (P ) is a p-group, then O p (G) < G. We also make use of the following renowned Zsigmondy's property as well as a particular consequence of it.
Lemma 2.4. [12] Let q be a natural number greater than 1. Then for every natural number m there exists a prime r such that r | q m − 1 but r q i − 1 for every 1 ≤ i ≤ m − 1 except for the following cases: (1) m = 6 and q = 2; (2) m = 2 and q = 2 l − 1 for some natural number l.
(3) If n = 2 then either q is a Mersenne prime and p = 2; or r is a Fermat prime and a = 1, or p = 3, a = 2 and q = 8.
The following theorem, which is pointed out in the introduction and is based on the Classification of the Finite Simple Groups, is essential in our proofs. Regarding coprime action, we just recall the following. Suppose that a finite group A acts coprimely on a finite group G. Then, for every prime p, there always exist an A-invariant Sylow p-subgroup in G and any two of them are conjugate by some element lying the fixed point subgroup C = C G (A). The same happens for p-complements when G is solvable. Also, every A-invariant subgroup obviously lies in a maximal A-invariant subgroup of G. We refer the non-familiarized reader to [8,Chapter 8] for a detailed presentation of the basic properties of coprime action. We only state here the two following known results.  Suppose that the action of a group A on a group V is coprime and that V is an elementary abelian p-group, with p prime. Then the action of A on V is semisimple.

Proofs
Proof of Theorem A. We argue by induction on |G|. Since the hypotheses are clearly inherited by A-invariant quotients of G, the inductive hypothesis implies that every proper A-invariant quotient of G is solvable. Let N be a minimal A-invariant normal subgroup of G. Of course, this includes the case N = G, that is, when G has no non-trivial proper A-invariant normal subgroup. Then, as G/N is solvable we only have to prove that N is solvable. Take p to be the largest prime divisor of |N | and let P be an A-invariant Sylow p-subgroup of N . By the Frattini argument, we have G = N N G (P ). Notice that if P G, it certainly follows that G is solvable, so we can assume that the A-invariant subgroup N G (P ) is contained in some maximal A-invariant subgroup, say U , of G. Then, by hypothesis, |G : U | = q a , with q prime and a ≤ 2. Observe that G = N U , so |G : U | = |N : N ∩ U | = q a . Since P ⊆ N ∩ U , then p = q, and by the choose of p, we have q < p. Moreover, as P is a Sylow p-subgroup of N and of N ∩ U , by Sylow theorem we get |N : N N (P )| ≡ 1 (mod p) and |N ∩ U : N N (P )| ≡ 1 (mod p). Both congruences imply that |N : N ∩ U | ≡ 1 (mod p) and this yields to two possibilities: q ≡ 1 (mod p) or q 2 ≡ 1 (mod p). As q < p, the first possibility cannot occur, whereas the second forces p to divide q + 1. This implies that p = q + 1, and hence, q = 2 and p = 3. Consequently, |N : N ∩ U | = q 2 = 4, so if L = Core N (N ∩ U ) we deduce that N/L can be embedded in the symmetric group S 4 . In particular, N/L is solvable. Now, we know that N is characteristically simple, so it is the direct product of isomorphic simple groups. From these facts we conclude that N is solvable, as required.
Proof of Theorem B. We induct on |G|. Then all proper A-invariant factors of G are supersolvable, so if N is a minimal A-invariant normal subgroup of G, we only need to prove that N has order p. Furthermore, we can assume that N is the only minimal Ainvariant normal subgroup of G. Indeed, if N and M are two of such subgroups, then G/(N ∩ M ) ∼ = G can be immersed in G/N × G/M , and hence, G would be supersolvable, so we have finished the proof.
Let p be the largest prime dividing |G| and let P be an A-invariant Sylow p-subgroup of G. First wet prove that P is elementary abelian and normal in G. Indeed, if P is not normal in G, then by hypothesis, there exists a maximal A-invariant subgroup U containing N G (P ) of prime index q ≡ 1 (mod p). This is not possible by the choice of p. Therefore, P G, and in particular, N ⊆ P . On the other hand, if we consider the Frattini subgroup Φ(G) of G, which trivially is A-invariant, we can assume that Φ(G) = 1. If not, we know that G/Φ(G) is supersolvable, and since the class of supersolvable groups is a saturated formation, it follows that G is supersolvable too. Thus, as Φ(P ) ≤ Φ(G) = 1, we conclude that P is an elementary abelian group, as wanted.
Now, by Theorem A we know that G is solvable, so by coprime action properties we can take an A-invariant p-complement H of G, and write G = P H. Notice that AH acts coprimely on the elementary abelian (normal) p-group P , and hence by applying Theorem 2.8, the action of AH on P is semisimple. This means that every AH-invariant subgroup of P has an AH-invariant complement in P . In particular, if N < P , then N has an AH-invariant complement T in P . Hence, as P is abelian, we deduce that T is an A-invariant normal subgroup of G, satisfying T ∩ N = 1. This clearly contradicts the existence of exactly one minimal A-invariant normal subgroup in G, which leads to N = P . Finally, we take a maximal A-invariant subgroup V such that H ⊆ V , which has index exactly p by hypothesis. As P V = G, then p = |G : V | = |P : P ∩ V |. However, P ∩ V V , and also, P ∩ V P because P is abelian, so the minimality of P implies that P ∩ V = 1. Consequently, |P | = p, and the proof is finished.
Proof of Theorem C. We argue by counterexample of minimal order. Let us take G and A satisfying the hypotheses with G non-solvable and |GA| as small as possible, where GA denotes the semidirect product of G by A. It is immediate to see that the hypotheses are inherited by A-invariant quotients of G, so for every non-trivial A-invariant normal subgroup N of G, we have that G/N is solvable. We can assume further that the action of A on G is non-trivial. Otherwise, the result follows by application of Lemma 2.1. Moreover, we may assume that A acts faithfully on G. On the contrary, we may consider A := A/C A (G) acting on G, where C A (G) denotes the kernel of action of A on G. By minimal counterexample, we also get that G is solvable.
We continue the proof by stating several steps. Take p to be the largest prime divisor of |G| and let P be an A-invariant Sylow p-subgroup of G. Observe that p ≥ 5 by application of Burnside's p a q b -theorem.
Step 1. We can assume that P is a non-maximal A-invariant subgroup of G.
Assume that the assertion is false. Then N G (P ) = P or G because N G (P ) is also Ainvariant. If N G (P ) = G, that is, if P G, as we have claimed in the first paragraph of the proof, G/P is solvable and thus, G is solvable too, a contradiction. Hence N G (P ) = P . Now, suppose first that Φ(P ) = 1 and we will get a contradiction. It is clear that Φ(P ) is a non-maximal A-invariant subgroup of G. By hypotheses, there exists a maximal A-invariant subgroup H of G such that Φ(P ) < H and |G : H| = r is a prime. Also, r ≤ p by our assumption on p. The action of G on the left cosets of H G (the core of H in G) is faithful, so we can assume that G/H G ≤ S r . If r = p, then P ≤ H G and thus P ≤ H. By the maximality of P , we have P = H G G, a contradiction. Therefore, |G : H| = p. On the other hand, notice that H G = 1, otherwise the fact that G ≤ S p implies that |P | = p, contradicting Φ(P ) = 1 . Then, by minimality, G/H G is solvable and H G must be non-solvable. Notice that P H G is an A-invariant subgroup of G with P H G = P , so we conclude that G = P H G by the maximality of P . This means that G/H G is a p-group, and we deduce that |G/H G | = p. It follows that H = H G G and G = P H. Let P 1 = P ∩ H, which is an A-invariant Sylow p-subgroup of H with P 1 P . Therefore, N G (P 1 ) = P N H (P 1 ). Since N G (P 1 ) is A-invariant and P ≤ N G (P 1 ), we have N H (P 1 ) = P 1 . By Lemma 2.2, O p (H) < H. Let T = O p (H). Then T G is A-invariant and G = P T . Let P 2 = T ∩ P . We claim that P 2 = 1. Otherwise, T is a p -subgroup. Note that P A ≤ GA, and we consider the action of P A on T . By [8, 8.2.3], we have that T has a P A-invariant Sylow p 1 -subgroup D for some prime p 1 dividing |T | . Hence DP ≤ G is A-invariant. Since T is non-solvable, we have D < T . So, in particular, we deduce that DP < G. However, by the maximality of P , we also know that DP = G, a contradiction. Thus, P 2 = 1, as claimed. Then P 2 P is A-invariant and N G (P 2 ) ≥ P . Again, the maximality of P implies that N G (P 2 ) = G or P . The first possibility cannot occur because A-invariant normal subgroups of G cannot be solvable. Thus, N G (P 2 ) = P and then, in particular, N T (P 2 ) is a p-subgroup. By Lemma 2.2, we conclude that O p (T ) < T . This certainly contradicts the definition of T .
The above paragraph shows that Φ(P ) = 1, or equivalently, that P is elementary abelian. Furthermore, we have P = N G (P ) = C G (P ). By [8,Theorem 7.2.1], it is known that G has a normal p-complement, say K. Since P is A-invariant, we obtain P A ≤ GA. Now, let us consider the action of P A on K. By applying [8,Theorem 8.2.3], we get that K has a P A-invariant l-subgroup L of K, where l is some prime divisor of |K|. Hence LP is A-invariant, so LP = G by the maximality of P . This demonstrates that G is solvable for being the product of two prime-power order subgroups. This is the final contradiction of this step.
Step 2. G has a unique minimal A-invariant normal subgroup N < G, which is non-solvable.
Suppose first that G has no non-trivial proper A-invariant normal subgroup. Then G is minimal normal subgroup of GA, which implies that G = G 1 × · · · × G s , where G i are isomorphic non-abelian simple groups. Let r be the largest prime divisor of |G| and let R be an A-invariant Sylow r-subgroup of G. By Step 1, there must exist some maximal A-invariant subgroup M of G such that |G : M | = t with t a prime. This gives G/M G ≤ S t and M G is an A-invariant normal subgroup of G. This forces that M G = 1 and thus, G ≤ S t . Note that t < r. This contradiction indicates that G must have a proper A-invariant normal subgroup.
If G has two different proper minimal A-invariant normal subgroups N 1 and N 2 , then G/N 1 and G/N 2 are solvable. Since N 1 ∩ N 2 = 1, then G is isomorphic to a subgroup of G/N 1 × G/N 2 , implying that G is solvable, a contradiction. Hence G has a unique proper minimal A-invariant normal subgroup, say N , with G/N solvable. This forces that N is not solvable, so the step is proved.
Step 3. Let q be the largest prime divisor of |N | and Q an A-invariant Sylow q-subgroup of N . Then N G (Q) is a maximal A-invariant subgroup of G.
Suppose first that N G (Q) = G. Then Q G is an A-invariant subgroup of G and Step 2 gives a contradiction. This shows that N G (Q) is a proper A-invariant subgroup of G.
Assume then that N G (Q) is a non-maximal A-invariant subgroup of G and we will get a contradiction. By the hypothesis, there is a maximal A-invariant subgroup U such that N G (Q) < U and |G : U | = p 2 is a prime. If U G = 1, then N ≤ U G . By the Frattini argument, it follows that G = N N G (Q) ≤ U , a contradiction. Hence U G = 1. So we have that the action of G on left cosets of U is faithful, and as a consequence, G ≤ S p 2 . Now, observe that G = N U , and thus, |G : U | = |N : N ∩ U | = p 2 , which leads to that p 2 is the largest prime divisor of |G| and p 2 | |N |. Therefore, p 2 = q. Moreover, |Q| = q and Q is a Sylow q-subgroup of G. However, |G : U | = q, which is a contradiction.
Step 4. Final contradiction. Let X = N G (Q) and Y = N N (Q). Then Y X and clearly Y = X. We prove that Y is not nilpotent; otherwise, N N (Q)/C N (Q) is a q-group, and by Lemma 2.2, we have O q (N ) < N , a contradiction. Then there exists an A-invariant Sylow s-subgroup S of Y such that S Y for some prime s | |Y |. By the Frattini argument, X = Y N X (S) and certainly N X (S) < X. Let T = N X (S), which is A-invariant. Again by the Frattini argument, we have G = N X = N T . Since T is non-maximal A-invariant in G, then by hypothesis, there is some maximal A-invariant subgroup M of G such that T < M with |G : M | = p 2 a prime. So we deduce that G = N M . This shows that M G = 1, that is, the action of G on the left cosets of M is faithful, so we have G ≤ S p 2 . Therefore, p 2 is the largest prime divisor of |G|. Note that |G : M | = |N : N ∩ M | = p 2 , so p 2 = q and hence, |Q| = q. Since N is non-solvable, it follows that N is a non-abelian simple group. By applying Lemma 2.3, we obtain that N G (Q) is a Frobenius group with cyclic q-complement and G is a simple group. As a consequence, G = N .
As we have proved that N is a simple group with a subgroup M ∩ N of index q, by Lemma 2.6, we have that N is one of the following groups: ( ( where n is some prime. It is well-known (see [1] for instance) that M 11 , M 23 , A q with q ≥ 5 and PSL 2 (11) do not admit non-trivial coprime automorphisms. This forces A to act trivially on any of them. We will prove that A also must act trivially on N in the remaining case, that is, in case (4). Recall that the order of the Singer cycle of PSL n (s) is equal to and it is an integer. This give rise to two possibilities: (n, s − 1) = q or 1. If (n, s − 1) = q, then q divides s − 1, and we next prove that this leads to a contradiction. Indeed, in this case we trivially have s = 2 and n = 2, so by Lemma 2.4, there must exist a prime dividing s n − 1 and not dividing s − 1. Necessarily, this prime must be q, a contradiction as wanted. Therefore, we may assume that (n, s − 1) = 1. We write s = r d , where r is a prime and d is a positive integer. By applying Lemma 2.5(2), we have either r ≡ 1 (mod n) or n = r = 2. The first possibility implies that n divides r d − 1, which contradicts our assumption. So we can assume that n = r = 2, and hence, again by Lemma 2.5(3), we get that q = s + 1 = 2 d + 1 is a Fermat prime, that is, d is a power of 2. This means that N = PSL 2 (2 d ), and in this case, it is known that Out(N ) ∼ = C d , with d a power of 2. As a consequence, N has no non-trivial coprime automorphisms, and so, A acts trivially on N too, or equivalently N ⊆ C, as we wanted to prove. Now, by Step 2, it trivially follows that C G (N ) = 1, so N ≤ G ≤ Aut(N ), that is, G is an almost simple group with socle N , where N is one of the groups appearing in the above list. However, in all cases, except case (2), we have that G/N is the trivial group or a cyclic 2-group. This occurs because the external automorphism groups of M 11 and M 23 are both trivial, Out(PSL 2 (11)) ∼ = C 2 , and Out(PSL 2 (2 d )) ∼ = C 2 d , with d a power of 2. Furthermore, cyclic 2-groups do not admit a non-trivial coprime action, since the automorphism group of a cyclic 2-group is a 2-group too, so in particular, C G/N (A) = G/N . Then, we can apply Lemma 2.7, and since we have proved that N ⊆ C, we deduce that G = C, that is, the action of A on G is trivial. Finally, for the remaining case, N ∼ = A q , it is known that Aut(A q ) = S q , so we have either G = A q or G = S q . But none of them admits a non-trivial coprime automorphism, and we conclude that in all cases the action of A on G is trivial. This is the final contradiction.