Matrix Multiplication

The most important—and initially, the most confusing—of all matrix operations.

If you've seen it before, you might remember a confusing rule about rows and columns, dot products, and a lot of finger-pointing. The "how" can seem strange and arbitrary.

The secret to understanding it is to ignore the "how" for a moment and focus on the "why." The "why" comes directly from our second view of matrices: a matrix is a transformation.

The "Why": Composing Transformations

What happens if you want to apply two transformations in a row? First, you rotate space 90 degrees counter-clockwise. Then, you shear space horizontally.

Let's call the rotation matrix RotRot and the shear matrix ShearShear. Applying RotRot to a vector vv gives you a new vector, let's call it v′v'. Then, applying ShearShear to v′v' gives you the final vector, v′′v''.

Shear⋅(Rot⋅v)=v′′Shear \cdot (Rot \cdot v) = v''

The central question of matrix multiplication is this: Is there a single, new matrix that represents this entire two-step process? Can we find a matrix MM that does the rotation *and then* the shear all in one go, such that M⋅v=v′′M \cdot v = v''?

Yes. That new matrix MM is the product of ShearShear and RotRot.

M=Shear⋅RotM = Shear \cdot Rot

Matrix multiplication is the composition of linear transformations. It's how we combine multiple transformations into a single, equivalent one.

Order Matters! AB≠BAAB \neq BA

Thinking about transformations makes it immediately obvious why the order of matrix multiplication is critical. Rotating then shearing gives you one result. Shearing then rotating gives you a completely different result! This is why, in general, ABAB is not the same as BABA. Matrix multiplication is not commutative.

The "How": The Row-Column Rule

The Dimension Rule

To multiply two matrices AA and BB to get C=ABC = AB, the inner dimensions must match.

If AA is m×nm \times n (m rows, n columns) and BB is n×pn \times p (n rows, p columns), then you can multiply them. The resulting matrix CC will have the outer dimensions: m×pm \times p.

(m×n)⋅(n×p)→(m×p)(m \times n) \cdot (n \times p) \rightarrow (m \times p)

The Calculation Rule

The entry in the ii-th row and jj-th column of the product matrix CC is the dot product of the ii-th row of AA and the jj-th column of BB.

Example:

A=[123456]A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}
B=[789101112]B = \begin{bmatrix} 7 & 8 \\ 9 & 10 \\ 11 & 12 \end{bmatrix}

AA is 2×32 \times 3, BB is 3×23 \times 2. The result CC will be 2×22 \times 2.

C11C_{11} (1st row, 1st col) = (Row 1 of A) · (Col 1 of B)

[1,2,3]⋅[7,9,11]=(1⋅7)+(2⋅9)+(3⋅11)=7+18+33=58[1, 2, 3] \cdot [7, 9, 11] = (1 \cdot 7) + (2 \cdot 9) + (3 \cdot 11) = 7 + 18 + 33 = 58

C12C_{12} (1st row, 2nd col) = (Row 1 of A) · (Col 2 of B)

[1,2,3]⋅[8,10,12]=(1⋅8)+(2⋅10)+(3⋅12)=8+20+36=64[1, 2, 3] \cdot [8, 10, 12] = (1 \cdot 8) + (2 \cdot 10) + (3 \cdot 12) = 8 + 20 + 36 = 64

C21C_{21} (2nd row, 1st col) = (Row 2 of A) · (Col 1 of B)

[4,5,6]⋅[7,9,11]=(4⋅7)+(5⋅9)+(6⋅11)=28+45+66=139[4, 5, 6] \cdot [7, 9, 11] = (4 \cdot 7) + (5 \cdot 9) + (6 \cdot 11) = 28 + 45 + 66 = 139

C22C_{22} (2nd row, 2nd col) = (Row 2 of A) · (Col 2 of B)

[4,5,6]⋅[8,10,12]=(4⋅8)+(5⋅10)+(6⋅12)=32+50+72=154[4, 5, 6] \cdot [8, 10, 12] = (4 \cdot 8) + (5 \cdot 10) + (6 \cdot 12) = 32 + 50 + 72 = 154

Result:

C=[5864139154]C = \begin{bmatrix} 58 & 64 \\ 139 & 154 \end{bmatrix}
Summary: The Most Important Operation
  • The "Why": Matrix multiplication is the composition of transformations.
  • The "How": The entry in row ii, column jj is the dot product of row ii of the first matrix and column jj of the second.
  • The Rule: The inner dimensions must match: (m×n)⋅(n×p)→(m×p)(m \times n) \cdot (n \times p) \rightarrow (m \times p).
  • The Warning: Order matters! In general, AB≠BAAB \neq BA.

Up Next: Now that we know the rules, let's look at some special "character" matrices—matrices with unique properties that play important roles in the story, like the Identity matrix and the Inverse matrix.