3 Induced representations 3.4 Example: D₄ to S₄ 3.6 Example: GL₂(F_p) — nonexaminable

3.5 Mackey’s formula — nonexaminable

We want to come up with a criterion for $\operatorname{Ind}_{H}^{G}\chi$ to be irreducible. This is equivalent to

\left\langle\operatorname{Ind}_{H}^{G}\chi,\operatorname{Ind}_{H}^{G}\chi% \right\rangle_{G}=1

which is in turn equivalent, by Frobenius reciprocity, to

\left\langle\operatorname{Res}^{G}_{H}\operatorname{Ind}_{H}^{G}\chi,\chi% \right\rangle_{H}=1.

So we try to solve the more general problem of decomposing $\operatorname{Res}^{G}_{H}\operatorname{Ind}_{H}^{G}V$ , as a representation of $H$ . In fact, we will solve the more general problem of finding

\operatorname{Res}^{G}_{H}\operatorname{Ind}_{K}^{G}V

for $H$ and $K$ arbitrary subgroups of $G$ .

3.5.1 Double cosets

Let $G$ be a group and let $H, K$ be subgroups. A double coset $H g K$ is a set of the form

HgK=\{hgk:h\in H,k\in K\}.

Like cosets, any two double cosets are either equal or disjoint, so they partition the group. Unlike cosets, they do not have to have the same size. A sequence $g_{1},\ldots,g_{r}$ such that $G$ is the disjoint union

G=\coprod_{i=1}^{r}Hg_{i}K

is called a set of double coset representatives for $H$ and $K$ in $G$ . We write $H\backslash G/K$ for the set of double cosets of $H$ and $K$ in $G$ .

Lemma 3.17.

The double cosets $H g K$ are the in bijection with the orbits of $H$ acting on $G/K$ .

Proof.

Two cosets $g K$ and $g^{\prime}K$ are in the same $H$ -orbit, if and only if $hgK=g^{\prime}K$ for some $h\in H$ , if and only if

HgK=Hg^{\prime}K.

So we have a bijection which takes the double coset $H g K$ to the $H$ -orbit of $g K$ . ∎

Example 3.18.

Let $H=K=S_{n-1}\subset G=S_{n}$ , $n\geq 3$ , thinking of $H$ as the set of elements that fix the number $n$ . Then $H=HeH$ is one double coset. I claim that $G=H\cup H(n-1\,n)H$ . Indeed, a complete set of left coset representatives for $H$ in $G$ is

\{e,(1n),(2n),\ldots,((n-1)n)\}.

As I can write $(in)=(i\,n-1)(n-1\,n)(i\,n-1)\in H(n-1\,n)H$ for $i\leq n-1$ , we see that the left cosets $(1n)H,\ldots,(n-1\,n)H$ are all contained in the same double coset $H(n-1\,n)H$ , as required.

This may be clearer from the orbits point of view. There is a bijection $G/H\to\{1,\ldots,n\}$ which respects the action of $G$ — the bijection takes $g H$ to $g(n)$ . The orbits of $H=S_{n-1}$ acting on $\{1,\ldots,n\}$ are $\{n\}$ and $\{1,\ldots,n-1\}$ , which gives the two double cosets.

If $s K$ is a left coset of $K$ , we write $H_{s}=H\cap sKs^{-1}$ ; note that this is independent of choice of representative $s$ . The notation is motivated by the fact that $H_{s}$ is the stabiliser of $s K$ for the action of $H$ on $G/K$ discussed above. This leads to the following lemma.

Lemma 3.19.

If $s\in G$ , then as a disjoint union of cosets of $K$ we have

HsK=h_{1}sK\cup h_{2}sK\ldots\cup h_{r}sK

where $h_{1},\ldots,h_{r}$ are a complete set of left coset representatives for $H_{s}$ in $H$ .

3.5.2 Mackey’s formula

Suppose that $K\subset G$ , that $s\in G$ , and that $(\rho,V)$ is a representation of $K$ . Then we can define a representation $({}^{s}\rho,V)$ of ${}^{s}H=sHs^{-1}$ by taking ${}^{s}V=V$ (same vector space) and ${}^{s}\rho(h)=\rho(s^{-1}hs)$ for $h\in{}^{s}H$ .

Theorem 3.20.

(Mackey’s formula) Let $H, K$ be subgroups of $G$ and let $(\rho,V)$ be a representation of $K$ . Let $s_{1},\ldots,s_{r}$ be a set of double coset representatives for $H$ and $K$ in $G$ .

Then

\operatorname{Res}^{G}_{H}\operatorname{Ind}_{K}^{G}V\cong\bigoplus_{s\in\{s_{% 1},\ldots,s_{r}\}}\operatorname{Ind}_{H_{s}}^{H}\operatorname{Res}^{{}^{s}K}_{% H_{s}}{}^{s}V.

Proof.

Let $g_{1},\ldots,g_{n}$ be the left coset representatives for $K$ in $G$ . Then

\operatorname{Ind}_{K}^{G}V=g_{1}V\oplus\ldots\oplus g_{n}V.

The action of $H$ permutes the $g_{i}V$ among themselves, with $g_{i}V$ and $g_{j}V$ in the same $H$ -orbit if and only if $Hg_{i}K=Hg_{j}K$ . Therefore for each double coset $H s K$ we have an $H$ -subrepresentation

W_{s}=\bigoplus_{i:g_{i}K\subset HsK}g_{i}V.

Now, by lemma 3.19 above, these $g_{i}$ have the form $h_{i}s$ for $h_{i}$ a set of left coset representatives for $H_{s}$ in $H$ . From the decomposition

W_{s}=\bigoplus_{h_{i}H_{s}\in H/H_{s}}h_{i}sV

we see that

W_{s}\cong\operatorname{Ind}_{H_{s}}^{H}sV.

It remains to show that $s V$ , as a representation of $H_{s}$ , is isomorphic to ${}^{s}V$ . The map ${}^{s}V\rightarrow sV$ taking $v$ to $s v$ is the required isomorphism, because

\rho(h)(sv)=\rho(h)\rho(s)v=\rho(s)\rho(s^{-1}hs)v=s{}^{s}\rho(h)v.

∎

3.5.3 Irreducibility criterion

Theorem 3.21.

Let $\rho$ be an irreducible representation of $H\subset G$ . Then $\operatorname{Ind}_{H}^{G}\rho$ is irreducible if and only if, for every $g$ in a set of double coset representatives of $H\backslash G/H$ , with $g\not\in H$ ,

\operatorname{Hom}_{H_{g}}(\rho,{}^{g}\rho)=0

where $H_{g}=H\cap gHg^{-1}$ .

Proof.

Indeed, let $S=g_{0},\ldots,g_{r}$ be such a set of double coset representatives, with $g_{0}\in H$ . By Mackey’s formula and Frobenius reciprocity,

	$\displaystyle\left\langle\operatorname{Ind}_{H}^{G}\rho,\operatorname{Ind}_{H}% ^{G}\rho\right\rangle_{G}$	$\displaystyle=\left\langle\rho,\operatorname{Res}^{G}_{H}\operatorname{Ind}_{H% }^{G}\rho\right\rangle_{H}$
		$\displaystyle=\left\langle\rho,\bigoplus_{g\in S}\operatorname{Ind}_{H_{g}}^{H% }{}^{g}\rho\right\rangle_{H}$
		$\displaystyle=\left\langle\rho,\rho\right\rangle_{H}+\sum_{g\in\{g_{1},\ldots,% g_{r}\}}\left\langle\rho,\operatorname{Ind}_{H_{g}}^{H}{}^{g}\rho\right\rangle% _{H}$
		$\displaystyle=\left\langle\rho,\rho\right\rangle_{H}+\sum_{g\in\{g_{1},\ldots,% g_{r}\}}\left\langle\rho,{}^{g}\rho\right\rangle_{H_{g}}.$

Since $\rho$ is irreducible, the first term on the right is equal to 1 and all the other terms are at least 0. So

\left\langle\operatorname{Ind}_{H}^{G}\rho,\operatorname{Ind}_{H}^{G}\rho% \right\rangle_{G}\geq 1

with equality if and only if $\operatorname{Hom}_{H_{g}}(\rho,{}^{g}\rho)=0$ for all $g\in S\setminus H$ . The result follows. ∎

Corollary 3.22.

If $H\lhd G$ and $\rho$ is an irreducible representation of $H$ , then

\operatorname{Ind}_{H}^{G}\rho

is irreducible if and only if $\rho\not\cong{}^{g}\rho$ for all $g$ in a set of coset representatives of $H$ in $G$ with $g\not\in H$ .

Proof.

If $H$ is normal, then double cosets, left cosets, and right cosets are all the same. For $g\in G$ , ${}^{g}\rho$ is another irreducible representation of $H$ so

\operatorname{Hom}_{H}(\rho,{}^{g}\rho)\neq 0

if and only if $\rho\not\cong{}^{g}\rho$ . ∎

3.5.4 Mackey examples

Example 3.23.

Let $G=D_{n}\supset H=C_{n}$ . Let $\chi:C_{n}\to\mathbb{C}^{\times}$ with $\chi(r)=\omega$ . The previous corollary shows that

\operatorname{Ind}_{H}^{G}\chi

is reducible if and only if $\chi\cong{}^{s}\chi$ . But

{}^{s}\chi(r)=\chi(s^{-1}rs)=\chi(r^{-1})=\omega^{-1}.

So $\operatorname{Ind}_{H}^{G}\chi$ is irreducible if and only if $\omega\neq\omega^{-1}$ (which we already knew).

Example 3.24.

Let $\rho$ be an irreducible representation of $S_{n-1}\subset S_{n}$ with character $\chi$ . Then $\{e,s\}$ is a set of double coset representatives for $S_{n-1}$ in $S_{n}$ , where $s=(n-1\,n)$ . We have

{}^{s}S_{n-1}\cap S_{n-1}=S_{n-2}

of elements fixing $n-1$ and $n$ .

Moreover, ${}^{s}\chi\cong\chi$ (because elements of $S_{n-2}$ are conjugate in $S_{n-2}$ if and only if they are conjugate in $S_{n-1}$ , so they have the same character!) and so we see

	$\displaystyle\left\langle\operatorname{Ind}_{S_{n-1}}^{S_{n}}\chi,% \operatorname{Ind}_{S_{n-1}}^{S_{n}}\chi\right\rangle$	$\displaystyle=\left\langle\chi,\chi\right\rangle_{S_{n-1}}+\left\langle\chi,% \chi\right\rangle_{S_{n-2}}$
		$\displaystyle=1+\left\langle\chi,\chi\right\rangle_{S_{n-2}}.$

For example, if $\chi$ is the degree 2 irreducible character of $S_{4}$ , then $\chi|_{S_{3}}$ is still irreducible so the right hand side of this formula gives two. And indeed,

\operatorname{Ind}_{S_{4}}^{S_{5}}\chi=\psi+\psi\epsilon

where $\psi$ is one of the irreducible five-dimensional representations of $S_{5}$ .

3.5.5 Index two

Let $G$ be a group and let $H$ be a subgroup of index two (necessarily normal). Let $s\in G$ with $s\not\in H$ . Let $\epsilon:G\to\mathbb{C}^{\times}$ be the unique (order two) character with kernel $H$ .

Of course, we have in mind the example $H=A_{n}$ , $G=S_{n}$ .

Theorem 3.25.

Suppose that $\rho$ is an irreducible representation of $H$ . Then

1.

if $\rho\cong{}^{s}\rho$ , then

$\operatorname{Ind}^{G}_{H}\rho\cong\sigma\oplus\epsilon\sigma$

for some irreducible representation $\sigma$ of $G$ with $\operatorname{Res}^{G}_{H}\sigma\cong\rho$ .
2.

if $\rho\not\cong{}^{s}\rho$ , then

$\sigma=\operatorname{Ind}_{H}^{G}\rho$

is irreducible, with $\sigma\cong\epsilon\sigma$ .

Proof.

Let $\rho$ have character $\chi$ . By the normal subgroup case of Mackey’s theorem,

\left\langle\operatorname{Ind}_{H}^{G}\chi,\operatorname{Ind}_{H}^{G}\chi% \right\rangle_{G}=1+\left\langle\chi,{}^{s}\chi\right\rangle_{H}.

It follows that the induction $\operatorname{Ind}_{H}^{G}\chi$ is irreducible if and only if $\chi\neq{}^{s}\chi$ . This gives the second statement, and most of the first (we leave the proof that, in all cases, $\operatorname{Ind}_{H}^{G}\chi\cong\epsilon\operatorname{Ind}_{H}^{G}\chi$ as an exercise.)

If $\chi={}^{s}\chi$ , then we have $\operatorname{Ind}_{H}^{G}\chi=\psi+\psi^{\prime}$ for some irreducible characters $\psi$ and $\psi^{\prime}$ of $G$ . Restricting to $H$ , we have $\psi|_{H}+\psi^{\prime}|_{H}=2\chi$ so that $\psi$ extends $\chi$ . But then we have

\operatorname{Ind}_{H}^{G}\chi=\operatorname{Ind}_{H}^{G}\operatorname{Res}^{G% }_{H}\psi=\psi(\operatorname{Ind}_{H}^{G}\mathbbm{1})=\psi(\mathbbm{1}+% \epsilon)=\psi+\epsilon\psi

so that $\operatorname{Ind}_{H}^{G}\chi=\psi+\epsilon\psi$ as required. ∎