Vector Dot Product Properties

In this article, we will learn different properties of dot product:

• Associative Property

• Distributive Property

• Commutative Property

TLDR

Just remember this:

• Vector dot product satisfies distributive property

• Generally, vector dot product doesn’t satisfy associativity

• Vector dot product is commutative

Distributive Property

So, distributive property means that a times the sum of b and c is equal to the sum of a times b and a times c. Let me give you two equations one only with scalar(like you learned in your high school) and then I’ll give you another example with vectors for linear algebra.

Scalar Distributive Property

\(a(b + c) = ab + ac\)

This is your normal distributive property where A times the sum of B and C is equal to A times B and a times C.

Vector Dot Product Distributive Property

\( \mathbf{a}^T(\mathbf{b} + \mathbf{c}) = \mathbf{a}^T\mathbf{b} + \mathbf{a}^T\mathbf{c} \)

This equation looks similar to the scalar distributive equation except one thing:

Here, we are using that transpose operation on vector A. For now, don’t worry about it but again remember what I said in our dot product article, whenever you see something like this, just know that this is just a dot product.

Ok, now let me prove this to you that the distributive property holds for vectors.

\(\mathbf{a}^T\mathbf{b} + \mathbf{a}^T\mathbf{c} = \sum_{i=1}^{n} a_i b_i + \sum_{i=1}^{n} a_i c_i = \sum_{i=1}^{n} \big(a_i b_i + a_i c_i\big) = \sum_{i=1}^{n} a_i (b_i + c_i) = \sum_{i=1}^{n} a_i (\mathbf{b} + \mathbf{c})_i = \mathbf{a}^T(\mathbf{b} + \mathbf{c}) \)

From our previous article, we know that:

\(\mathbf{a}^T\mathbf{b} = \sum_{i=1}^{n} a_i b_i \)

So, we can say:

\(\mathbf{a}^T\mathbf{b} + \mathbf{a}^T\mathbf{c} = \sum_{i=1}^{n} a_i b_i + \sum_{i=1}^{n} a_i c_i\)

Now, the dimensionality and the vector A, so let’s take them as common:

\( \sum_{i=1}^{n} \big(a_i b_i + a_i c_i\big) = \sum_{i=1}^{n} a_i (b_i + c_i) = \sum_{i=1}^{n} a_i (\mathbf{b} + \mathbf{c})_i \)

Finally, you’ll get:

\(\sum_{i=1}^{n} a_i (\mathbf{b} + \mathbf{c})_i = \mathbf{a}^T(\mathbf{b} + \mathbf{c})\)

Associative Property

So, associative property means that a times b times c is equal to a times the product of b and c, which is also equal to the product of a and b times c. Let me give you two equations one only with scalar(like you learned in your high school) and then I'll give you another example with vectors for linear algebra.

Scalar Associative Property

\(a(b.c)=(a.b)c\)

The associative property basically means you can put paratheses wherever you want if you’re multiplying all the scalar values or summing all scalar values.

Vector Dot Product Associative Property

The dot product of 2 vectors is not associative (there are some special cases where it does hold):

\(\mathbf{a}^T(\mathbf{b}^T\mathbf{c})\;\neq\; (\mathbf{a}^T\mathbf{b})^T\mathbf{c}\)

ok, lets see a proof to understand why the dot product of two vectors is not associative.

Case 1 (a, b and c are completely different vectors and none of these vectors is a vector with all 0s)

Note that:

\(\underbrace{\mathbf{b}^T\mathbf{c}}{\text{scalar}} \quad \text{and} \quad \underbrace{\mathbf{a}^T\mathbf{b}}{\text{scalar}}\)

The Dot product of vector b and c is a scalar. The Dot product of vector a and b is a scalar.

Now, if we are getting 2 distinct scalars from these 2 operations and both these scalars are getting multiplied with two different vectors:

\(\mathbf{a}^T(\mathbf{b}^T\mathbf{c})\quad \quad (\mathbf{a}^T\mathbf{b})^T\mathbf{c} \)

Vector A is getting multiplied with a different scalar and vector C is getting multiplied with a different scalar.

So, 2 distinct vectors are getting multiplied with two distinct scalars. So of course, the result will not be the same.

Also, both of these vectors won’t even have a similar orientation:

Vector A will be a row vector and vector C will be a column vector because we are transposing vector A but not C. Hence C stays as column vector but vector A changes to a row vector.

If you are confused about this orientation reasoning, then don’t worry, it’ll be clearer once we get to transpose concept.

Case 2: Dimensionality Differs between the 3 vectors

Let’s assume vector a has 5 elements hence its dimensionality is 5.

Let’s assume vector b and c has 3 elements, hence their dimensionality is 3.

You can realize that we the equation:

\((\mathbf{a}^T\mathbf{b})^T\mathbf{c}\)

is invalid because we can’t do the dot product of two vectors a and b with different dimensionality.

Commutative Property

Scalar Commutative Property

\( ab = ba\)

In scalar, commutative property holds true. Commutative property in scalar means:

A times B is equal to B times A.

Vector Dot Product Commutative

Dot product of two vectors is commutative, let me prove that to you:

\(\mathbf{a}^T\mathbf{b} = \sum_{i=1}^{n} a_i b_i = \sum_{i=1}^{n} b_i a_i = \sum_{i=1}^{n} b_i a_i = \mathbf{b}^T\mathbf{a}\)

From our previous article, we know that:

\(\mathbf{a}^T\mathbf{b} = \sum_{i=1}^{n} a_i b_i \)

We can also write this as:

\(\sum_{i=1}^{n} a_i b_i = \sum_{i=1}^{n} b_i a_i\)

As each corresponding element of a and b is a scalar so they are commutative and we can switch their places.

Finally, we can rewrite the equation to this:

\(\sum_{i=1}^{n} b_i a_i = \mathbf{b}^T\mathbf{a}\)

Hence, dot product of two vectors (considering same dimensionality, this is a basic requirement to do dot product operation) is commutative.

Now, lets do all this in python and then we will wrap up

Python

Lots of equations and proofs but I think this might help you understand concepts in depth and even if you forget, you can prove the properties in a minute.

Distributive Property

# create random vectors
n = 10
a = np.random.randn(n)
b = np.random.randn(n)
c = np.random.randn(n)

# the two results
res1 = np.dot( a , (b+c) )
res2 = np.dot(a,b) + np.dot(a,c)

# compare them
print([ res1,res2 ])

Here, I’m creating three random vectors a, b, and c with 10 elements each.

Then I’m computing the left side of our distributive property equation and storing it in res1.

After that, I’m computing the right side and storing it in res2.

If you print both results, you’ll see they’re exactly the same (or very close due to floating point precision). This proves that the distributive property holds for vectors.

Associative Property

# create random vectors
n = 5
a = np.random.randn(n)
b = np.random.randn(n)
c = np.random.randn(n)

# the two results
res1 = np.dot( a , np.dot(b,c) )
res2 = np.dot( np.dot(a,b) , c )

# compare them
print(res1)
print(res2)

Here, I’m creating three random vectors a, b, and c with 5 elements each.

Then I’m computing the left side of associative property and storing it in res1.

After that, I’m computing the right side and storing it in res2.

If you print both results, you’ll see they’re different! This proves that the associative property doesn’t hold for dot product of vectors.

But wait, there are some special cases where associative property works:

1. When one vector is the zeros vector

2. When a==b==c (all three vectors are the same)

In the next article, we will learn about vector length and after that we will finally understand the geometric interpretation of dot product. Till then, Goodbye!