Matrix products

A matrix product contains the dot product of each left-hand row with each right-hand column.

2 min read1 figureSources & further reading

On this page 8 sections

Overview
Many inputs through one matrix
Complete the product
Every row must fit every column
Scores for observations
Key takeaway
Sources & further reading
Concept connections

01Many inputs through one matrix#

You already know how a matrix acts on one vector. A matrix product applies the left matrix to each column of the right matrix. Collect the resulting output vectors as columns of the product.

For $C=AB$ , entry $C_{ij}$ is output $i$ for input column $j$ . It therefore uses row $i$ of $A$ and column $j$ of $B$ .

Figure 1A has rows (1,2) and (3,0). B has rows (2,4) and (1,−1). Highlighted row 1 of A and column 2 of B give entry C12=1×4+2×(−1)=2 in C=AB. Question marks mark other entries not yet computed.

In the diagram, $C_{12}$ pairs row $(1,2)$ with column $(4,-1)$ . The contributions are $4$ and $-2$ , so $C_{12}=2$ . The question marks stand for entries still to compute.

02Complete the product#

Keep $A$ with rows $(1,2),(3,0)$ and $B$ with rows $(2,4),(1,-1)$ . Column 1 of $B$ is $(2,1)$ . Applying $A$ gives $(4,6)$ , so that is column 1 of $C$ .

Column 2 of $B$ is $(4,-1)$ . Applying $A$ gives $(2,12)$ . Put both output columns together:

C=\begin{pmatrix}4&2\\6&12\end{pmatrix}

Check your reasoning

A has rows $(1,0)$ , $(2,1)$ ; B has rows $(3,2)$ , $(-1,0)$ . For $C=AB$ , compute $C_{21}$ .

Show answer and explanation

Row 2 dots column 1: $2\cdot3+1\cdot(-1)=5$ .

03Every row must fit every column#

If $A$ is $m\times d$ and $B$ is $d\times k$ , each row-column pair has $d$ products to add. There are $m$ left rows and $k$ right columns, giving $mk$ output entries.

C_{ij}=\sum_{r=1}^{d}A_{ir}B_{rj}

Check your reasoning

A is $3\times2$ ; B is $2\times1$ . A learner says AB is $2\times2$ . Repair its shape.

A $2\times2$
B $3\times1$
C $1\times3$

Show answer and explanation

$3\times1$

Three A rows and one B column give a $3\times1$ result.

Matching dimensions make the multiplication possible; they do not justify multiplying corresponding matrix entries. Even for two square matrices, each output entry generally uses an entire row and column.

04Scores for observations#

Let a data matrix $X$ have rows $(1,2),(2,0),(0,1)$ . Let $W$ have columns $(2,1)$ and $(-1,3)$ , the weights of two scoring rules. Each entry of $XW$ scores one observation under one rule.

XW=\begin{pmatrix}4&5\\4&-2\\1&3\end{pmatrix}

For example, row 2 under rule 2 is $2(-1)+0(3)=-2$ . Changing row 2 of $X$ affects row 2 of the output. Changing column 2 of $W$ affects column 2 of the output. Reading those roles is a useful check on an implementation.

Check your reasoning

A has score rows P $(1,0)$ , Q $(2,-1)$ . B has observation columns U $(3,2)$ , V $(1,4)$ . Find score Q for observation V.

Show answer and explanation

-2

Dot Q with V: $2-4=-2$ .

Key takeaway

For entry (i,j), dot row i of the left matrix with column j of the right matrix.

Compute the entries of a compatible matrix product.

Sources & further reading

[1]
Goodfellow, Bengio and Courville: Deep Learning, 2.1–2.2 ↗www.deeplearningbook.org · Article
[2]
MIT 18.06SC: Lecture 3, Multiplication and Inverse Matrices ↗ocw.mit.edu · Article
[3]
Boyd and Vandenberghe, Introduction to Applied Linear Algebra ↗stanford.edu · Article

01Many inputs through one matrix#

02Complete the product#

03Every row must fit every column#

04Scores for observations#

For entry (i,j), dot row i of the left matrix with column j of the right matrix.

Sources & further reading

Reference this concept