Matrix Operations

The Rules of Moving in Space

In the last lesson, we introduced the two powerful views of a matrix. Before we can fully harness the power of transformations, we need to get comfortable with the basic "grammar" of matrices.

How do we add, subtract, and scale them? These operations are simple and intuitive, closely mirroring the vector operations we've already learned.

The Ground Rules: Matrix Dimensions

Before we can perform any operation, we have to talk about the shape or dimensions of a matrix. The dimensions are always given as rows x columns.

A matrix with 3 rows and 4 columns is a $3 \times 4$ matrix.
A matrix with 1 row and 5 columns is a $1 \times 5$ matrix (which is also a row vector!).

This is crucial because most matrix operations have strict rules about the dimensions of the matrices involved.

Addition and Subtraction

You can only add or subtract two matrices if they have the exact same dimensions. To perform the operation, you simply add or subtract the corresponding elements in each position.

Let $A$ be a $2 \times 2$ matrix and $B$ be a $2 \times 2$ matrix.

A = \begin{bmatrix} 1 & -2 \\ 3 & 4 \end{bmatrix}

B = \begin{bmatrix} 5 & 0 \\ 1 & -3 \end{bmatrix}

Addition ( $A + B$ ):

A + B = \begin{bmatrix} 1+5 & -2+0 \\ 3+1 & 4-3 \end{bmatrix} = \begin{bmatrix} 6 & -2 \\ 4 & 1 \end{bmatrix}

Scalar Multiplication

To multiply a matrix by a scalar, you multiply every single element inside the matrix by that scalar.

Let's take our matrix $A$ and multiply it by the scalar 3.

3 \times A = 3 \times \begin{bmatrix} 1 & -2 \\ 3 & 4 \end{bmatrix} = \begin{bmatrix} 3 & -6 \\ 9 & 12 \end{bmatrix}

Geometric Intuition: If a matrix represents a transformation, multiplying it by a scalar `c` scales the entire transformation.

The Transpose: Flipping the Grid

To find the transpose of a matrix, you flip it across its main diagonal. The rows become the columns, and the columns become the rows. The transpose of a matrix $A$ is written as $A^T$ .

Let's take a $2 \times 3$ matrix $C$ .

C = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}

To find $C^T$ , the first row becomes the first column, and the second row becomes the second column.

C^T = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}

Notice that the dimensions also flip. A $2 \times 3$ matrix becomes a $3 \times 2$ matrix.

Why is the Transpose So Important?

The transpose is the key to unlocking the Four Fundamental Subspaces. It also appears constantly in key formulas, like the Normal Equations for linear regression ( $A^TA\hat{x} = A^Tb$ ) and in the definition of a symmetric matrix ( $A = A^T$ ).

Summary: The Basic Toolkit

Addition/Subtraction: Add/subtract element-wise. Matrices must have the same dimensions.
Scalar Multiplication: Multiply every element by the scalar.
Transpose ( $A^T$ ): Flip the matrix along its diagonal. The rows become columns.

The Two Views of a Matrix

Matrix Multiplication