Entrywise Products

The Hadamard product multiplies equal-shaped matrices entry by entry.

2 min read0 figuresSources & further reading

On this page 7 sections

Overview
Multiply matching entries
Keep the operation straight
Match shapes and positions
Key takeaway
Sources & further reading
Concept connections

01Multiply matching entries#

The Hadamard product multiplies equal-shaped matrices entry by entry. It keeps their shape. In ML, this operation can apply a supplied mask or gate separately to each coordinate.

(A\odot B)_{ij}=A_{ij}B_{ij}.

Suppose $A$ has rows $(2,-1),(3,0)$ and $B$ has rows $(4,5),(0,-2)$ . The four products are $8,-5,0,0$ , so $A\odot B$ has rows $(8,-5),(0,0)$ .

02Keep the operation straight#

Each output entry uses exactly one entry from each input. No row-column sum appears. A row-by-column product $AB$ is a different operation, with a different shape rule and generally different numbers.

Check your reasoning

Rows of A: (2,−1),(0,3). Rows of B: (4,2),(5,−2). Rows of A ⊙ B?

A(8,−2),(0,−6)
B(3,6),(15,−6)
C(8,2),(0,6)

Show answer and explanation

(8,−2),(0,−6)

Multiply corresponding entries and retain their positions.

A mask $M$ containing zeros and ones sets an entry to zero when the corresponding mask is zero and preserves it when the mask is one. For row $(6,-3,2)$ and mask $(1,0,1)$ , the result is $(6,0,2)$ .

03Match shapes and positions#

A $2\times3$ matrix can have a Hadamard product with another $2\times3$ matrix. Equal numbers of entries are insufficient if their shapes differ. Reshaping or broadcasting is a separate convention that must be stated explicitly.

Check your reasoning

A draft computes an entry of A ⊙ B by summing a left row against a right column. What should change?

ASum only the diagonal
BMultiply just the two matching entries
CTranspose the final matrix

Show answer and explanation

Multiply just the two matching entries

The draft used ordinary matrix multiplication.

Changing one entry of $B$ affects only its matching entry in $A\odot B$ . If the matching $A$ entry is zero, the output stays zero. Position carries the meaning of this operation.

Key takeaway

Compute the Hadamard product of two equal-shaped real matrices.

Compute the Hadamard product of two equal-shaped real matrices.

Sources & further reading

[1]
Boyd and Vandenberghe, Introduction to Applied Linear Algebra ↗stanford.edu · Article
[2]
MIT 6.390, Appendix A: Matrix Calculus ↗introml.mit.edu · Article