Linear Combinations and Span

What Can You Build With Vectors?

We've assembled our toolkit. We have our fundamental objects (vectors) and the rules for manipulating them (addition and scalar multiplication).

Now, we can ask the most important creative question in all of linear algebra: With a given set of vectors, what can we build?

The answer to this question lies in two beautifully linked concepts: Linear Combinations and Span.

Linear Combinations: The Recipe
A linear combination is the fundamental "recipe" for building new vectors out of old ones. It's an expression made from two ingredients: Scalar Multiplication (scaling your ingredient vectors) and Vector Addition (mixing them together).

Let's say we have two vectors, v=[1,2]v = [1, 2] and w=[−3,1]w = [-3, 1].

A linear combination is any vector that can be written in the form:

c1v1+c2v2+⋯+cnvnc_1v_1 + c_2v_2 + \dots + c_nv_n

Where c1,c2,…c_1, c_2, \dots are any scalars (numbers) you choose. Let's compute one:

2v+1w=2[12]+1[−31]=[24]+[−31]=[−15]2v + 1w = 2\begin{bmatrix} 1 \\ 2 \end{bmatrix} + 1\begin{bmatrix} -3 \\ 1 \end{bmatrix} = \begin{bmatrix} 2 \\ 4 \end{bmatrix} + \begin{bmatrix} -3 \\ 1 \end{bmatrix} = \begin{bmatrix} -1 \\ 5 \end{bmatrix}

So, the vector [−1,5][-1, 5] is one possible linear combination of vv and ww. It's one of the things we can "build."

Span: All the Things You Can Possibly Build
The set of all possible vectors you can create through a linear combination of a set of vectors is called the span of those vectors.

If we only had vector vv, its span would be the infinite line it sits on. But with both vv and ww, which point in different directions, we can reach any point in the entire 2D plane by choosing different scalars.

The span of vv and ww is all of 2D space (written as R2\mathbb{R}^2).

What if our vectors are collinear?

If we chose v=[1,2]v = [1, 2] and u=[2,4]u = [2, 4], notice that uu is just 2v2v. No matter what scalars we choose for c1v+c2uc_1v + c_2u, we can't escape the line that both vectors sit on. The span is just a line, not the full plane.

The Core Idea for Quants & ML

Span tells you the "expressive power" of your features. Imagine your model is `Stock Price = c₁ * (GDP Growth) + c₂ * (Interest Rate)`. The span of the "GDP" and "Interest Rate" vectors represents all possible outcomes your model can explain. If you add a new factor that is just a multiple of GDP growth, you haven't expanded the span at all. You've added redundant information.

Summary: The Creative Tools
  • Linear Combination: The recipe for building new vectors. It's a weighted sum of a set of "ingredient" vectors (c1v1+c2v2c_1v_1 + c_2v_2).
  • Span: The result of all possible recipes. It's the complete set of all vectors that can be built from your starting set.

Up Next: We'll formalize the idea of "redundant" vectors with the concept of **Linear Independence**, one of the most important theoretical pillars of the entire subject.