In mathematics, a generalized flag variety (or simply flag variety) is a homogeneous space whose points are flags in a finite-dimensional vector space V over a field F. When F is the real or complex numbers, a generalized flag variety is a smooth or complex manifold, called a real or complex flag manifold. Flag varieties are naturally projective varieties.

Flag varieties can be defined in various degrees of generality. A prototype is the variety of complete flags in a vector space V over a field F, which is a flag variety for the special linear group over F. Other flag varieties arise by considering partial flags, or by restriction from the special linear group to subgroups such as the symplectic group. For partial flags, one needs to specify the sequence of dimensions of the flags under consideration. For subgroups of the linear group, additional conditions must be imposed on the flags.

In the most general sense, a generalized flag variety is defined to mean a projective homogeneous variety, that is, a smooth projective variety X over a field F with a transitive action of a reductive group G (and smooth stabilizer subgroup; that is no restriction for F of characteristic zero). If X has an F-rational point, then it is isomorphic to G/P for some parabolic subgroup P of G. A projective homogeneous variety may also be realised as the orbit of a highest weight vector in a projectivized representation of G. The complex projective homogeneous varieties are the compact flat model spaces for Cartan geometries of parabolic type. They are homogeneous Riemannian manifolds under any maximal compact subgroup of G, and they are precisely the coadjoint orbits of compact Lie groups.

Flag manifolds can be symmetric spaces. Over the complex numbers, the corresponding flag manifolds are the Hermitian symmetric spaces. Over the real numbers, an R-space is a synonym for a real flag manifold and the corresponding symmetric spaces are called symmetric R-spaces.

A flag in a finite dimensional vector space V over a field F is an increasing sequence of subspaces, where "increasing" means each is a proper subspace of the next (see filtration):

${\displaystyle \{0\}=V_{0}\subset V_{1}\subset V_{2}\subset \cdots \subset V_{k}=V.}$

If we write the dim V_i = d_i then we have

${\displaystyle 0=d_{0}<d_{1}<d_{2}<\cdots <d_{k}=n,}$

where n is the dimension of V. Hence, we must have k ≤ n. A flag is called a complete flag if d_i = i for all i, otherwise it is called a partial flag. The signature of the flag is the sequence (d₁, ..., d_k).

A partial flag can be obtained from a complete flag by deleting some of the subspaces. Conversely, any partial flag can be completed (in many different ways) by inserting suitable subspaces.

According to basic results of linear algebra, any two complete flags in an n-dimensional vector space V over a field F are no different from each other from a geometric point of view. That is to say, the general linear group acts transitively on the set of all complete flags.

Fix an ordered basis for V, identifying it with Fⁿ, whose general linear group is the group GL(n,F) of n × n invertible matrices. The standard flag associated with this basis is the one where the ith subspace is spanned by the first i vectors of the basis. Relative to this basis, the stabilizer of the standard flag is the group of nonsingular lower triangular matrices, which we denote by B_n. The complete flag variety can therefore be written as a homogeneous space GL(n,F) / B_n, which shows in particular that it has dimension n(n−1)/2 over F.

Note that the multiples of the identity act trivially on all flags, and so one can restrict attention to the special linear group SL(n,F) of matrices with determinant one, which is a semisimple algebraic group; the set of lower triangular matrices of determinant one is a Borel subgroup.

If the field F is the real or complex numbers we can introduce an inner product on V such that the chosen basis is orthonormal. Any complete flag then splits into a direct sum of one-dimensional subspaces by taking orthogonal complements. It follows that the complete flag manifold over the complex numbers is the homogeneous space

${\displaystyle U(n)/T^{n}}$

where U(n) is the unitary group and Tⁿ is the n-torus of diagonal unitary matrices. There is a similar description over the real numbers with U(n) replaced by the orthogonal group O(n), and Tⁿ by the diagonal orthogonal matrices (which have diagonal entries ±1).

The partial flag variety

${\displaystyle F(d_{1},d_{2},\ldots d_{k},\mathbb {F} )}$

is the space of all flags of signature (d₁, d₂, ... d_k) in a vector space V of dimension n = d_k over F. The complete flag variety is the special case that d_i = i for all i. When k=2, this is a Grassmannian of d₁-dimensional subspaces of V.

This is a homogeneous space for the general linear group G of V over F. To be explicit, take V = Fⁿ so that G = GL(n,F). The stabilizer of a flag of nested subspaces V_i of dimension d_i can be taken to be the group of nonsingular block lower triangular matrices, where the dimensions of the blocks are n_i := d_i − d_i−1 (with d₀ = 0).

Restricting to matrices of determinant one, this is a parabolic subgroup P of SL(n,F), and thus the partial flag variety is isomorphic to the homogeneous space SL(n,F)/P.

If F is the real or complex numbers, then an inner product can be used to split any flag into a direct sum, and so the partial flag variety is also isomorphic to the homogeneous space

${\displaystyle U(n)/U(n_{1})\times \cdots \times U(n_{k})}$

in the complex case, or

${\displaystyle O(n)/O(n_{1})\times \cdots \times O(n_{k})}$

in the real case.

The upper triangular matrices of determinant one are a Borel subgroup of SL(n,F), and hence the stabilizers of partial flags are parabolic subgroups. Furthermore, a partial flag is determined by the parabolic subgroup which stabilizes it.

Hence, more generally, if G is a semisimple algebraic or Lie group, then the (generalized) flag variety for G is G/P where P is a parabolic subgroup of G. The correspondence between parabolic subgroups and generalized flag varieties allows each to be understood in terms of the other.

The extension of the terminology "flag variety" is reasonable, because points of G/P can still be described using flags. When G is a classical group, such as a symplectic group or orthogonal group, this is particularly transparent. If (V, ω) is a symplectic vector space then a partial flag in V is isotropic if the symplectic form vanishes on proper subspaces of V in the flag. The stabilizer of an isotropic flag is a parabolic subgroup of the symplectic group Sp(V,ω). For orthogonal groups there is a similar picture, with a couple of complications. First, if F is not algebraically closed, then isotropic subspaces may not exist: for a general theory, one needs to use the split orthogonal groups. Second, for vector spaces of even dimension 2m, isotropic subspaces of dimension m come in two flavours ("self-dual" and "anti-self-dual") and one needs to distinguish these to obtain a homogeneous space.

If G is a compact, connected Lie group, it contains a maximal torus T and the space G/T of left cosets with the quotient topology is a compact real manifold. If H is any other closed, connected subgroup of G containing T, then G/H is another compact real manifold. (Both are actually complex homogeneous spaces in a canonical way through complexification.)

The presence of a complex structure and cellular (co)homology make it easy to see that the cohomology ring of G/H is concentrated in even degrees, but in fact, something much stronger can be said. Because G → G/H is a principal H-bundle, there exists a classifying map G/H → BH with target the classifying space BH. If we replace G/H with the homotopy quotient G_H in the sequence G → G/H → BH, we obtain a principal G-bundle called the Borel fibration of the right multiplication action of H on G, and we can use the cohomological Serre spectral sequence of this bundle to understand the fiber-restriction homomorphism H*(G/H) → H*(G) and the characteristic map H*(BH) → H*(G/H), so called because its image, the characteristic subring of H*(G/H), carries the characteristic classes of the original bundle H → G → G/H.

Let us now restrict our coefficient ring to be a field k of characteristic zero, so that, by Hopf's theorem, H*(G) is an exterior algebra on generators of odd degree (the subspace of primitive elements). It follows that the edge homomorphisms

${\displaystyle E_{r+1}^{0,r}\to E_{r+1}^{r+1,0}}$

of the spectral sequence must eventually take the space of primitive elements in the left column H*(G) of the page E₂ bijectively into the bottom row H*(BH): we know G and H have the same rank, so if the collection of edge homomorphisms were not full rank on the primitive subspace, then the image of the bottom row H*(BH) in the final page H*(G/H) of the sequence would be infinite-dimensional as a k-vector space, which is impossible, for instance by cellular cohomology again, because a compact homogeneous space admits a finite CW structure.

Thus the ring map H*(G/H) → H*(G) is trivial in this case, and the characteristic map is surjective, so that H*(G/H) is a quotient of H*(BH). The kernel of the map is the ideal generated by the images of primitive elements under the edge homomorphisms, which is also the ideal generated by positive-degree elements in the image of the canonical map H*(BG) → H*(BH) induced by the inclusion of H in G.

The map H*(BG) → H*(BT) is injective, and likewise for H, with image the subring H*(BT)^W(G) of elements invariant under the action of the Weyl group, so one finally obtains the concise description

${\displaystyle H^{*}(G/H)\cong H^{*}(BT)^{W(H)}/{\big (}{\widetilde {H}}^{*}(BT)^{W(G)}{\big )},}$

where ${\displaystyle {\widetilde {H}}^{*}}$ denotes positive-degree elements and the parentheses the generation of an ideal. For example, for the complete complex flag manifold U(n)/Tⁿ, one has

${\displaystyle H^{*}{\big (}U(n)/T^{n}{\big )}\cong \mathbb {Q} [t_{1},\ldots ,t_{n}]/(\sigma _{1},\ldots ,\sigma _{n}),}$

where the t_j are of degree 2 and the σ_j are the first n elementary symmetric polynomials in the variables t_j. For a more concrete example, take n = 2, so that U(2)/[U(1) × U(1)] is the complex Grassmannian Gr(1,${\displaystyle \mathbb {C} }$²) ≈ ${\displaystyle \mathbb {C} }$P¹ ≈ S². Then we expect the cohomology ring to be an exterior algebra on a generator of degree two (the fundamental class), and indeed,

${\displaystyle H^{*}{\big (}U(2)/T^{2}{\big )}\cong \mathbb {Q} [t_{1},t_{2}]/(t_{1}+t_{2},t_{1}t_{2})\cong \mathbb {Q} [t_{1}]/(t_{1}^{2}),}$

as hoped.

If G is a semisimple algebraic group (or Lie group) and V is a (finite dimensional) highest weight representation of G, then the highest weight space is a point in the projective space P(V) and its orbit under the action of G is a projective algebraic variety. This variety is a (generalized) flag variety, and furthermore, every (generalized) flag variety for G arises in this way.

Armand Borel showed^{[citation needed]} that this characterizes the flag varieties of a general semisimple algebraic group G: they are precisely the complete homogeneous spaces of G, or equivalently (in this context), the projective homogeneous G-varieties.

Let G be a semisimple Lie group with maximal compact subgroup K. Then K acts transitively on any conjugacy class of parabolic subgroups, and hence the generalized flag variety G/P is a compact homogeneous Riemannian manifold K/(K∩P) with isometry group K. Furthermore, if G is a complex Lie group, G/P is a homogeneous Kähler manifold.

Turning this around, the Riemannian homogeneous spaces

M = K/(K∩P)

admit a strictly larger Lie group of transformations, namely G. Specializing to the case that M is a symmetric space, this observation yields all symmetric spaces admitting such a larger symmetry group, and these spaces have been classified by Kobayashi and Nagano.

If G is a complex Lie group, the symmetric spaces M arising in this way are the compact Hermitian symmetric spaces: K is the isometry group, and G is the biholomorphism group of M.

Over the real numbers, a real flag manifold is also called an R-space, and the R-spaces which are Riemannian symmetric spaces under K are known as symmetric R-spaces. The symmetric R-spaces which are not Hermitian symmetric are obtained by taking G to be a real form of the biholomorphism group G^c of a Hermitian symmetric space G^c/P^c such that P := P^c∩G is a parabolic subgroup of G. Examples include projective spaces (with G the group of projective transformations) and spheres (with G the group of conformal transformations).

• Parabolic Lie algebra
• Bruhat decomposition

• Robert J. Baston and Michael G. Eastwood, The Penrose Transform: its Interaction with Representation Theory, Oxford University Press, 1989.
• Jürgen Berndt, Lie group actions on manifolds, Lecture notes, Tokyo, 2002.
• Jürgen Berndt, Sergio Console and Carlos Olmos, Submanifolds and Holonomy, Chapman & Hall/CRC Press, 2003.
• Michel Brion, Lectures on the geometry of flag varieties, Lecture notes, Varsovie, 2003.
• James E. Humphreys, Linear Algebraic Groups, Graduate Texts in Mathematics, 21, Springer-Verlag, 1972.
• S. Kobayashi and T. Nagano, On filtered Lie algebras and geometric structures I, II, J. Math. Mech. 13 (1964), 875–907, 14 (1965) 513–521.