Basis and Dimension

The Perfect Building Blocks

We've spent the last two lessons developing two critical ideas: Span (what we can build) and Linear Independence (are our building blocks redundant?).

Now, we combine these two ideas to answer the ultimate question: What is the perfect, most efficient set of building blocks for a given space? The answer is a basis.

What is a Basis?

A basis for a vector space is a set of vectors that satisfies two conditions simultaneously:

The set must be linearly independent. (No redundant vectors.)
The span of the set must be the entire space. (The vectors must be powerful enough to build everything.)

A basis is the goldilocks set of vectors—not too few and not too many. It is the minimal set of vectors required to describe an entire space.

The Standard Basis: The Simplest Example

In the 2D xy-plane, the standard basis vectors are $i = [1, 0]$ and $j = [0, 1]$ . They are linearly independent and they span all of $\mathbb{R}^2$ . Thus, they form a basis.

Many Bases, One Space

A crucial point: a vector space can have infinitely many different bases. For example, the vectors $v = [1, 2]$ and $w = [-3, 1]$ are also linearly independent and span $\mathbb{R}^2$ , so they too form a valid basis.

Dimension: The Unchanging Number

This is the beautiful, unifying idea. The number of vectors in any basis for a vector space is always the same. This unique number is called the dimension of the space.

Any basis for the 2D plane ( $\mathbb{R}^2$ ) will have exactly two vectors. Therefore, $\mathbb{R}^2$ has a dimension of 2.
Any basis for 3D space ( $\mathbb{R}^3$ ) will have exactly three vectors. Therefore, $\mathbb{R}^3$ has a dimension of 3.

The Core Idea for Quants & ML

When we talk about **Dimensionality Reduction**, we are talking about finding a lower-dimensional subspace that captures most of the important information in our high-dimensional data.

Imagine a dataset of stocks with 50 features (50 dimensions). **Principal Component Analysis (PCA)** is an algorithm that finds a new, more efficient **basis** for this data. By projecting your data onto the first few vectors of this new basis, you reduce the dimension from 50 to maybe 3 or 4, making your models simpler and more robust.

Summary: The Essential Framework

Basis: A set of vectors that is both linearly independent and **spans the entire space**.
Dimension: The one thing all bases for a space share is the **number of vectors** they contain. This number is the dimension of the space.

Lesson 1.10: Linear Independence

Lesson 1.12: Vector Spaces and Subspaces