Steinitz exchange lemma

The Steinitz exchange lemma is a theorem in linear algebra concerning bases, dimensionality of a vector space, stating that for any set smaller than a spanning set, there is a set of vectors in the spanning set but missing from the smaller set that can be added to the smaller set to make that set spanning as well.

It can be used, for example, to show that any two bases for a finite-dimensional vector space have the same number of elements. The result is named after the German mathematician Ernst Steinitz. The result is often called the Steinitz–Mac Lane exchange lemma, also recognizing the generalization¹ by Saunders Mac Lane of Steinitz's lemma to matroids.²

Let ${\displaystyle U}$ and ${\displaystyle W}$ be finite subsets of a vector space ${\displaystyle V}$. If ${\displaystyle U}$ is a set of linearly independent vectors, and ${\displaystyle W}$ spans ${\displaystyle V}$, then:

1. ${\displaystyle |U|\leq |W|}$;

2. There is a set ${\displaystyle W'\subseteq W}$ with ${\displaystyle |W'|=|W|-|U|}$ such that ${\displaystyle U\cup W'}$ spans ${\displaystyle V}$.

Suppose ${\displaystyle U=\{u_{1},\dots ,u_{m}\}}$ and ${\displaystyle W=\{w_{1},\dots ,w_{n}\}}$. We wish to show that ${\displaystyle m\leq n}$, and that after rearranging the ${\displaystyle w_{j}}$ if necessary, the set ${\displaystyle \{u_{1},\dotsc ,u_{m},w_{m+1},\dotsc ,w_{n}\}}$ spans ${\displaystyle V}$. We proceed by induction on ${\displaystyle m}$.

For the base case, suppose ${\displaystyle m}$ is zero. In this case, the claim holds because there are no vectors ${\displaystyle u_{i}}$, and the set ${\displaystyle \{w_{1},\dotsc ,w_{n}\}}$ spans ${\displaystyle V}$ by hypothesis.

For the inductive step, assume the proposition is true for ${\displaystyle m-1}$. By the inductive hypothesis we may reorder the ${\displaystyle w_{i}}$ so that ${\displaystyle \{u_{1},\ldots ,u_{m-1},w_{m},\ldots ,w_{n}\}}$ spans ${\displaystyle V}$. Since ${\displaystyle u_{m}\in V}$, there exist coefficients ${\displaystyle \mu _{1},\ldots ,\mu _{n}}$ such that

${\displaystyle u_{m}=\sum _{i=1}^{m-1}\mu _{i}u_{i}+\sum _{j=m}^{n}\mu _{j}w_{j}}$.

At least one of the ${\displaystyle \mu _{j}}$ for ${\displaystyle j\geq m}$ must be non-zero, since otherwise this equality would contradict the linear independence of ${\displaystyle \{u_{1},\ldots ,u_{m}\}}$; this also shows that indeed ${\displaystyle m\leq n.}$ By reordering ${\displaystyle \mu _{m}w_{m},\ldots ,\mu _{n}w_{n}}$ if necessary, we may assume that ${\displaystyle \mu _{m}}$ is nonzero. Therefore, we have

${\displaystyle w_{m}={\frac {1}{\mu _{m}}}\left(u_{m}-\sum _{j=1}^{m-1}\mu _{j}u_{j}-\sum _{j=m+1}^{n}\mu _{j}w_{j}\right)}$.

In other words, ${\displaystyle w_{m}}$ is in the span of ${\displaystyle \{u_{1},\ldots ,u_{m},w_{m+1},\ldots ,w_{n}\}}$. Since this span contains each of the vectors ${\displaystyle u_{1},\ldots ,u_{m-1},w_{m},w_{m+1},\ldots ,w_{n}}$, by the inductive hypothesis it contains ${\displaystyle V}$.

The Steinitz exchange lemma is a basic result in computational mathematics, especially in linear algebra and in combinatorial algorithms.³