Vector Multiplication Explained: The Outer Product

I’ve mentioned previously that there are several way to multiply two vectors and today we are going to learn another method called “the outer product”.

Let me start by explaining the notation and how its different from the dot product:

Lets say we have two vectors v and w. Vector v is (1,n) dimensional and Vector w is (n,1) dimensional.

Let’s first look at the dot product. When we find the dot product of these two vectors, we will get a scalar value.

\(\underbrace{\mathbf{v}^T}{1 \times n} \underbrace{\mathbf{w}}{n \times 1} = \underbrace{s}_{1 \times 1 \text{ (scalar)}}\)

Now, lets learn about outer product. In outer product, we transpose the w vector and not the v vector (It’s a subtle difference but once we get to matrix multiplication, you’ll clearly understand about this subtle change). The outer product results in a matrix.

\( \underbrace{\mathbf{v}}{n \times 1} \underbrace{\mathbf{w}^T}{1 \times m} = \underbrace{\mathbf{M}}_{n \times m \text{ (matrix)}}\)

Unlike the dot product, the outer product can be computed between two vectors of different lengths. But the above notation still assumes that both of the vectors are column vectors.

Let me explain how you should think about outer product and get an intuitive explanation of its output matrix:

Let’s take these two vectors for example:

\( \mathbf{v} = \begin{bmatrix} 2 \\ a \\ -3 \\ b \end{bmatrix}, \quad \mathbf{w} = \begin{bmatrix} 5 \\ c \\ 1 \end{bmatrix} \)

To find the outer product, we need to transpose the w vector and then multiply them:

\(\mathbf{v} \mathbf{w}^T = \begin{bmatrix} 2 \\ a \\ -3 \\ b \end{bmatrix} \begin{bmatrix} 5 & c & 1 \end{bmatrix} = \begin{bmatrix} 2 \cdot 5 & 2 \cdot c & 2 \cdot 1 \\ a \cdot 5 & a \cdot c & a \cdot 1 \\ -3 \cdot 5 & -3 \cdot c & -3 \cdot 1 \\ b \cdot 5 & b \cdot c & b \cdot 1 \end{bmatrix}\)

Think of this multiplication operation in terms of column:

• First, you are scaling the vector v by 5

• Then, you are scaling the vector v by c

• Finally you are scaling the vector v by 1.

The output is a matrix that keeps all these different scaled vectors.

Intuitively, this is why we don’t need both vectors to be of same dimension because one of them is being used to scale the other vector

You can also think of this operation row wise, but I would stick with column perspective as that’s what we are primarily focusing on.

Outer Product with Python

Now, lets do it in python and then we will wrap up.

Let’s start by importing the usual stuff:

import numpy as np
import matplotlib.pyplot as plt

v1 = np.array([  1, 2, 3 ])
v2 = np.array([ -1, 0, 1 ])

# outer product
np.outer(v1,v2)

We created two vectors and then just use the outer() built-in function from numpy and you’ll get the outer product of the two vectors.

In the next one, we will cover the vector cross product. Till then, Goodbye!