The Column Space & Rank

The World of Possible Outputs

For the past several lessons, our focus has been entirely on finding a solution $x$ to the equation $Ax = b$ . We've built an incredible algorithmic machine—Gaussian Elimination—to do this.

Now, it's time to change our perspective.

Let's stop thinking about a specific $b$ and start thinking about all possible $b$ 's. When we use our matrix $A$ as a transformation, what is the entire universe of possible output vectors? Where can our input vectors possibly land?

The answer to this question is a beautiful, fundamental concept: the Column Space.

Defining the Column Space

The Column Space of a matrix

A

, written as

C(A)

, is the set of all possible linear combinations of its column vectors. In other words, **the Column Space is the span of the columns of

A

**.

Remember the "Column Picture" of $Ax = b$ ? We saw that the product $Ax$ is simply a linear combination of the columns of A, where the weights are the entries of the vector $x$ .

x_1 \cdot (\text{col } 1) + x_2 \cdot (\text{col } 2) + \dots + x_n \cdot (\text{col } n) = Ax

The Key Insight

The Column Space of

A

is the set of all possible output vectors

b

. The equation

Ax = b

has a solution if, and only if, the vector

b

lives inside the Column Space of

A

. If

b

is outside

C(A)

, the system is inconsistent.

Visualizing the Column Space

Let's take a 3x2 matrix

A

A = \begin{bmatrix} 1 & -1 \\ 2 & 0 \\ 3 & 1 \end{bmatrix}

The columns are $c_1 = [1, 2, 3]$ and $c_2 = [-1, 0, 1]$ . These are two vectors in 3D space.

The Column Space of $A$ is the **span** of these two vectors.
Since $c_1$ and $c_2$ are not collinear, their span is a **plane** passing through the origin in 3D space.
This means if we take *any* 2D input vector $x$ , the output $Ax$ will *always* land somewhere on this specific plane.
If we pick a target vector $b$ that lies on this plane, $Ax=b$ has a solution. If $b$ is off the plane, there is no solution.

Finding a Basis for the Column Space

The columns of

A

span the Column Space, but they might be linearly dependent. We need a non-redundant basis. The pivot columns of the original matrix

A

form this basis.

Example:

Let's find a basis for $C(A)$ for this matrix $A$ :

A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 2 & 4 & 7 & 9 \\ 3 & 6 & 8 & 11 \end{bmatrix}

Step 1: Reduce A to Row Echelon Form (REF).

After elimination ( $R_2 \to R_2 - 2R_1$ , $R_3 \to R_3 - 3R_1$ , then $R_3 \to R_3 + R_2$ ), we get the REF matrix $U$ :

U = \begin{bmatrix} \mathbf{1} & 2 & 3 & 4 \\ 0 & 0 & \mathbf{1} & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}

Step 2: Identify the pivot columns in `U`.

The pivots (the first non-zero entries in each row) are in **Column 1** and **Column 3**.

Step 3: The basis is the corresponding columns from the original matrix `A`.

A basis for $C(A)$ is:

\left\{ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}, \begin{bmatrix} 3 \\ 7 \\ 8 \end{bmatrix} \right\}

Warning

Do NOT use the pivot columns from `U`. Row operations change the column space. You must use the columns from the original `A`. The REF simply tells you *which* columns to pick.

The Concept of Rank

The essential, "true" dimension of the Column Space is called the **rank** of the matrix.

The **rank** of a matrix $A$ is the **dimension of its Column Space**.

Equivalently, and more practically: The **rank of $A$ is the number of pivots** in its row echelon form.

In our example, we found 2 pivots, so the **rank of $A$ is 2**. This means the four column vectors of $A$ , which live in 3D space, actually only span a 2-dimensional subspace (a plane).

Up Next: We've answered the question, "What are all the possible outputs?" Now we'll ask the opposite, "What inputs get mapped to zero?" This will lead us to the second of the four fundamental subspaces: the incredibly important **Null Space**.