Matrix Operations

The Rules of Moving in Space

In the last lesson, we introduced the two powerful views of a matrix: as a container for data and as a transformation of space. Before we can fully harness the power of transformations, we need to get comfortable with the basic "grammar" of matrices.

How do we add, subtract, and scale them? These operations are simple and intuitive, closely mirroring the vector operations we've already learned. They are the essential groundwork for everything that follows.

The Ground Rules: Matrix Dimensions

Before we can perform any operation, we have to talk about the shape or dimensions of a matrix. The dimensions are always given as rows x columns.

  • A matrix with 3 rows and 4 columns is a 3×43 \times 4 matrix.
  • A matrix with 1 row and 5 columns is a 1×51 \times 5 matrix (which is also a row vector!).

This is crucial because most matrix operations have strict rules about the dimensions of the matrices involved.

Addition and Subtraction
This is the easiest operation, and it works exactly like vector addition. You can only add or subtract two matrices if they have the exact same dimensions. To perform the operation, you simply add or subtract the corresponding elements in each position.

Example:

Let AA be a 2×22 \times 2 matrix and BB be a 2×22 \times 2 matrix.

A=[1−234]A = \begin{bmatrix} 1 & -2 \\ 3 & 4 \end{bmatrix}
B=[501−3]B = \begin{bmatrix} 5 & 0 \\ 1 & -3 \end{bmatrix}
Addition (A+BA + B):
A+B=[1+5−2+03+14−3]=[6−241]A + B = \begin{bmatrix} 1+5 & -2+0 \\ 3+1 & 4-3 \end{bmatrix} = \begin{bmatrix} 6 & -2 \\ 4 & 1 \end{bmatrix}
Subtraction (A−BA - B):
A−B=[1−5−2−03−14−(−3)]=[−4−227]A - B = \begin{bmatrix} 1-5 & -2-0 \\ 3-1 & 4-(-3) \end{bmatrix} = \begin{bmatrix} -4 & -2 \\ 2 & 7 \end{bmatrix}

If we tried to add AA to a 3×23 \times 2 matrix, the operation would be undefined.

Scalar Multiplication
This also works exactly like it does for vectors. To multiply a matrix by a scalar, you multiply every single element inside the matrix by that scalar.

Example:

Let's take our matrix AA and multiply it by the scalar 3.

3×A=3×[1−234]=[3×13×(−2)3×33×4]=[3−6912]3 \times A = 3 \times \begin{bmatrix} 1 & -2 \\ 3 & 4 \end{bmatrix} = \begin{bmatrix} 3 \times 1 & 3 \times (-2) \\ 3 \times 3 & 3 \times 4 \end{bmatrix} = \begin{bmatrix} 3 & -6 \\ 9 & 12 \end{bmatrix}

Geometric Intuition: If a matrix represents a transformation, multiplying it by a scalar `c` scales the entire transformation.

The Transpose: Flipping the Grid
The transpose is a simple but incredibly useful operation. To find the transpose of a matrix, you flip it across its main diagonal. The rows become the columns, and the columns become the rows. The transpose of a matrix AA is written as ATA^T.

Example:

Let's take a 2×32 \times 3 matrix CC.

C=[123456]C = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}

To find CTC^T, the first row [1,2,3][1, 2, 3] becomes the first column. The second row [4,5,6][4, 5, 6] becomes the second column.

CT=[142536]C^T = \begin{bmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{bmatrix}

Notice that the dimensions also flip. A 2×32 \times 3 matrix becomes a 3×23 \times 2 matrix.

Why is the Transpose So Important?

The transpose is the key to unlocking the Four Fundamental Subspaces. It also appears constantly in key formulas, like the Normal Equations for linear regression (ATAx^=ATbA^TA\hat{x} = A^Tb) and in the definition of a symmetric matrix (A=ATA = A^T).

Summary: The Basic Toolkit
  • Addition/Subtraction: Add/subtract element-wise. Matrices must have the same dimensions.
  • Scalar Multiplication: Multiply every element by the scalar.
  • Transpose (ATA^T): Flip the matrix along its diagonal. The rows become columns.

These are the simple rules. But what about multiplying a matrix by another matrix? That's a completely different beast, and the subject of our next, very important lesson.