Characteristic submonoid of group not implies subgroup

Statement
It is possible to have a group $$G$$ and a submonoid $$M$$ of $$G$$ such that $$\sigma(M) = M$$ for every automorphism $$\sigma$$ of $$G$$ such that $$M$$ is not a subgroup of $$M$$. (By submonoid, we mean a subset that is a monoid with the binary operation and identity element inherited from the whole group).

Similar facts

 * Characteristic submonoid of nilpotent group not implies subgroup

Opposite facts

 * Nonempty finite subsemigroup of group is subgroup: Any finite nonempty subsemigroup of a group is a subgroup. More generally, for a periodic group, i.e., a group where every element has finite order, any nonempty subsemigroup is a subgroup.
 * Nonempty characteristic subsemigroup of abelian group implies subgroup: For an abelian group, any characteristic submonoid, and more generally, any nonempty characteristic subsemigroup, must be a subgroup. This is because the inverse map is an automorphism for Abelian groups.

Facts used

 * 1) uses::Characteristically simple and abelian implies characteristic in holomorph: If $$A$$ is a characteristically simple Abelian group, then $$A$$ is characteristic in its holomorph: the semidirect product of $$A$$ and $$\operatorname{Aut}(A)$$.
 * 2) uses::Automorphism group action lemma for quotients: Suppose $$A$$ is an abelian normal subgroup of a group $$G$$, and $$\sigma$$ is an automorphism of $$G$$ that restricts to an automorphism $$\alpha$$ of $$A$$ and descends to an automorphism $$\sigma'$$ of $$G/A$$. Then, if $$\rho:G/A \to \operatorname{Aut}(A)$$ denotes the map by the conjugation action of the quotient (note: this requires abelianness of the subgroup), then $$\rho \circ \sigma' = c_\alpha \circ \rho$$.

Example involving the affine group
Consider the group $$G = GA(1,\mathbb{Q})$$ (see general affine group:GA(1,Q)): the general affine group over the rational numbers. this group can be described concretely in many ways:


 * It is the semidirect product of the additive group of rational numbers by the multiplicative group (i.e., it is the holomorph of the additive group of rational numbers).
 * It is the group (under composition) of all linear maps $$x \mapsto ax + b$$ from $$\mathbb{Q}$$ to itself, with $$a \in \mathbb{Q} \setminus 0$$ and $$b \in \mathbb{Q}$$.
 * It is the group of upper triangular $$2 \times 2$$ invertible matrices over the rationals, where both diagonal entries are equal.

Define $$A$$ as the following subgroup:


 * It is the normal subgroup comprising the additive group of rational numbers: the base of the semidirect product.
 * It is the subgroup comprising the translation maps: $$x \mapsto x+ b$$.
 * It is the group of upper triangular $$2 \times 2$$ matrices with $$1$$s on the diagonal.

A lemma
Given: $$G = GA(1,\mathbb{Q})$$, $$A$$ is described as above, $$\sigma$$ is an automorphism of $$G$$.

To prove: $$\sigma$$ sends every coset of $$A$$ in $$G$$ to itself.

An immediate corollary of the above is that every union of cosets of $$A$$ in $$G$$ is a characteristic subset of $$G$$.

Based on the above, we can construct many examples of characteristic submonoids of $$G$$ that are not subgroups. The idea is to choose any submonoid of $$\mathbb{Q}^* \cong G/A$$, then take its inverse image in $$G$$ under the quotient map to get the characteristic submonoid of $$G$$. Some examples are below.

Contraction mapping submonoid
Define $$M$$ as the following submonoid (described in all the alternate descriptions):


 * It is the subset of $$G$$ comprising those elements whose multiplicative group coordinate has modulus at most $$1$$.
 * It is the set of all linear maps of the form $$x \mapsto ax + b$$ where $$0 < |a| \le 1$$ and $$a,b \in \mathbb{Q}$$.(in other words, it is those linear maps that are contractions).
 * It is the set of all upper triangular $$2 \times 2$$ invertible matrices over the rationals where the two diagonal entries are equal and have absolute value at most $$1$$.

Clearly, $$M$$ is a submonoid of $$G$$ -- this follows from the fact that the set of $$a$$ for which $$0 < |a| \le 1$$ form a submonoid of $$\mathbb{Q}^*$$ under multiplication. On the other hand, $$M$$ is not a subgroup of $$G$$. By the preceding observations, $$M$$ is a characteristic submonoid of $$G$$ that is not a subgroup.

Nonnegative powers of 2 mapping submonoid

 * It is the subset of $$G$$ comprising those elements whose multiplicative group coordinate is a nonnegative integer power of 2.
 * It is the set of all linear maps of the form $$x \mapsto 2^nx + b$$ where $$n$$ is a nonnegative integer.
 * It is the set of all upper triangular $$2 \times 2$$ invertible matrices over the rationals where the two diagonal entries are equal to each other and their value is of the form $$2^n$$, $$n$$ a nonnegative integer.

Clearly, $$M$$ is a submonoid of $$G$$ -- this follows from the fact that the set of nonnegative integer powers of 2 is a submonoid of $$\mathbb{Q}^*$$. On the other hand, $$M$$ is not a subgroup of $$G$$. By the preceding observations, $$M$$ is a characteristic submonoid of $$G$$ that is not a subgroup.

Nilpotent group example
It is possible to construct a nilpotent group (in fact, a nilpotent group in which every automorphism is inner) and a characteristic submonoid of the group (in fact, one that is contained in the center) that is not a subgroup.