The Column Space & Rank

The World of Possible Outputs

Let's stop thinking about a specific $b$ and start thinking about all possible $b$ 's. When we use our matrix $A$ as a transformation, what is the entire universe of possible output vectors? Where can our input vectors possibly land?

The answer to this question is a beautiful, fundamental concept: the Column Space.

Defining the Column Space

The Column Space of a matrix $A$ , written as $C(A)$ , is the set of all possible linear combinations of its column vectors. In other words, **the Column Space is the span of the columns of $A$ **.

Remember the "Column Picture" of $Ax = b$ ? The product $Ax$ is simply a linear combination of the columns of A, where the weights are the entries of the vector $x$ .

x_1 \cdot (\text{col } 1) + x_2 \cdot (\text{col } 2) + \dots + x_n \cdot (\text{col } n) = Ax

The Core Insight

The Column Space of

A

is the set of all possible output vectors

b

. The equation

Ax = b

has a solution if, and only if, the vector

b

lives inside the Column Space of

A

. If

b

is outside

C(A)

, the system is inconsistent.

Finding a Basis for the Column Space

The columns of

A

span the Column Space, but they might be linearly dependent. We need a non-redundant basis. The pivot columns of the original matrix

A

form this basis.

Step 1: Reduce A to Row Echelon Form (REF)

For matrix `A`, we find its REF, which we'll call `U`:

A = \begin{bmatrix} 1 & 2 & 3 & 4 \\ 2 & 4 & 7 & 9 \\ 3 & 6 & 8 & 11 \end{bmatrix} \quad \xrightarrow{\text{Elim}} \quad U = \begin{bmatrix} \mathbf{1} & 2 & 3 & 4 \\ 0 & 0 & \mathbf{1} & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}

Step 2: Identify the pivot columns in `U`

The pivots (the first non-zero entries in each row) are in Column 1 and Column 3.

Step 3: The basis is the corresponding columns from the original matrix `A`

A basis for $C(A)$ is:

\left\{ \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}, \begin{bmatrix} 3 \\ 7 \\ 8 \end{bmatrix} \right\}

Warning

Do NOT use the pivot columns from `U`. Row operations change the column space. You must use the columns from the original `A`. The REF simply tells you *which* columns to pick.

The Concept of Rank

The **rank** of a matrix $A$ is the **dimension of its Column Space**.

Equivalently, and more practically: The **rank of $A$ is the number of pivots** in its row echelon form.

In our example, we found 2 pivots, so the **rank of $A$ is 2**. This means the four column vectors of $A$ , which live in 3D space, actually only span a 2-dimensional subspace (a plane).

Lesson 2.5: LU Decomposition

Lesson 2.7: The Null Space