Python and NumPy: Multiplication of Scalars, Vectors and Matrices

Perquisites

This uses the numerical Python library which has to be imported before running any scripts. As this library is commonly referenced, by convention it is imported as a two letter abbreviation np.

Python

Element by Element Multiplication or Array Multiplication

There are two different ways to multiply Scalars, Vectors and Matrices:

  • Element by Element Multiplication (multiply)
    • Known as multiply and can be carried out using the function np.multiply(a,b) or a*b
    • Each corresponding element of the first array multiplies the corresponding element of the second array.
    • As a consequence has the requirement that both arrays are equally sized.
  • Array Multiplication (dot product)
    • Known as the dot product and can be carried out using the function np.dot(a,b) or a@b
    • The inner dimensions of the first array (number of columns) must match the inner dimensions of the second array (number of rows)
    • Special Case: Python by default treats vectors as having a single axis and displays them as a row or column as it sees fit. As a result there are two additional functions inner(a,b) and outer(a,b) which perform the inner and outer dot product of two equally sized vectors. The function a@b can be used directly if a and b are explicitly reshaped using np.reshape.

Scalars

Starting off we a scalar these may appear to at first glance look identical. For instance if we purchase a single object type such as 5 pens from a single store at a fixed price of £2. Then the total spend per item type is £10 and the total spend on all items is also £10 because we only purchased one item type.

Python
q=[5]
p=[2]
cost per item type=[10]
cost per item type=[10]
total cost(all items)=10
total cost(all items)=10

The results of all 2 values are the same 10 as expected. However under closer observation the cost per item is enclosed in a [ ] and the total cost isn’t which shows that there is a difference between multiply ( * ) and dot ( @ ). This difference corresponds to dimensionality and can be observed when we move to more complicated objects such as vectors.

Vectors

When we move to a Vector we can immediately see the difference between multiply (*) which gives the total spent on each item type and dot (@) which gives the total spent.

The first method (element by element multiplication) in mathematics would be written out as:

\displaystyle \underset{{1\text{ by }2}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & 3 \end{array}} \right]}}\,*\underset{{1\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 2 & 6 \end{array}} \right]}}\,=\underset{{1\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {5\times 2} & {3\times 6} \end{array}} \right]}}\,=\underset{{1\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {10} & {18} \end{array}} \right]}}\,

The second method (array multiplication) in mathematics would be written out as:

\displaystyle \underset{{1\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & 3 \end{array}} \right]}}\,\text{@}\underset{{2\text{ by 1}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 2 \\ 6 \end{array}} \right]}}\,=\underset{{1\text{ by 1}}}{\mathop {\left[ {5\times 2+3\times 6} \right]}}\,=\underset{{1\text{ by 1}}}{\mathop {\left[ {28} \right]}}\,

q on the left hand side has 1 row by 2 columns and p has 2 rows by 1 column. For vector multiplication to take place, the inner dimensions must match i.e. columns of q and rows of p have to be equal and this condition is satisfied.

Python
q=[5 3]
p=[2 6]
cost per item type=[10 18]
cost per item type=[10 18]
total cost(all items)=28
total cost(all items)=28

Special Case: Outer Dot Product

For element by element multiplication, the order doesn’t matter i.e. p*q is the same as q*p. However for array multiplication, the order does matter and the inner dimensions have to match.

In the above we looked at the case where q was a row vector and p was a column vector and in this case the inner dimensions were taken to be the matching lengths of the two vectors.

\displaystyle \underset{{1\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & 3 \end{array}} \right]}}\,\text{@}\underset{{2\text{ by 1}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 2 \\ 6 \end{array}} \right]}}\,=\underset{{1\text{ by 1}}}{\mathop {\left[ {5\times 2+3\times 6} \right]}}\,=\underset{{1\text{ by 1}}}{\mathop {\left[ {28} \right]}}\,

Let’s contrast this with the case when q is the column vector and p is the row vector. In this case, the outer dimensions will be the lengths of q and p respectively and the inner dimensions will match at a value of 1 allowing array multiplication to take place and yielding a square output matrix.

In this case q is a column vector and p is a column vector and in this case the inner dimensions were taken to be the matching lengths of the two vectors.

\displaystyle \underset{{2\text{ by 1}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 \\ 3 \end{array}} \right]}}\,\text{@}\underset{{1\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 2 & 6 \end{array}} \right]}}\,=\underset{{\text{2 by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {5\times 2} & {5\times 6} \\ {3\times 2} & {3\times 6} \end{array}} \right]}}\,=\underset{{\text{2 by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {10} & {30} \\ 6 & {18} \end{array}} \right]}}\,

Python displays vectors as both rows and columns, depending on what is most convenient. Let’s take the earlier created p and look at it’s value and we’ll see it as a row.

Now let’s double click on in to view it in more detail and we’ll see instead that it is listed as a column vector:

So you may be tempted to ask, is p a row vector or is it a column vector? If we look at the transpose of p:

Python

We see that pt is identical to p…

As we seen above p is neither a row or a column vector.

When using vectors with array multiplication it is recommended to explicitly express them as a row or column and we can do this using the function np.reshape. In our case we will make q a column vector and p a row vector. To perform element by element multiplication, we must transpose one of these, so the dimensions match (otherwise multiply * performs the same operation as dot @)

q=[[2]
 [6]]
p=[[3 5]]
cost per item type=[[ 6]
 [30]]
cost per item type=[[ 6]
 [30]]
total cost(all items)=[[ 6 10]
 [18 30]]
total cost(all items)=[[ 6 10]
 [18 30]]
Python

When viewed in the variable explorer we get p with dimensions (1, 2) opposed to (2,) and q with dimensions (2, 1) opposed to (2,)

The function q@p or np.dot(q,p) will assign q as a row and p as a column by default with the inner dimensions being the lengths of the vectors and the outer dimensions being 1. As a consequence there are two other dot functions, np.outer(q,p) which will instead explicitly perform the multiplication with the length of the vectors being the outer dimensions of the square matrix and the additional function np.inner(q,p) which will explicitly perform the multiplication with the length of the vectors being the inner dimensions and yielding a 1 by 1 scalar result.

Python
q=[2 6]
p=[3 5]
multiply=[ 6 30]
multiply=[ 6 30]
dot product=36
dot product=36
inner dot product=36
outer dot product=[[ 6 10]
 [18 30]]

Vectors and Matrices

The functions inner and outer only work on vectors, they do not work on matrices. When applied to a matrix they will instead convert the matrix into a vector by reshaping it into a single dimension adding each row together.

However these functions do not need to be applied to a dot product involving a matrix as matrices have a specific number of rows and a specific number of columns. Moreover, if the case v@m, the vector v will be used as a row vector in an attempt to match the number of columns of the row vector with the number of rows of the matrix m.

\displaystyle \underset{{\text{1 by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & 6 \end{array}} \right]}}\,\text{@}\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,=\underset{{\text{1 by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {5\times 1+6\times 3} & {5\times 2+6\times 4} \end{array}} \right]}}\,=\underset{{\text{1 by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {23} & {34} \end{array}} \right]}}\,

Conversely in the case m@v the vector will be used as a column vector so the number of columns in the matrix m match the number rows of the column vector v:

\displaystyle \underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,\text{@}\underset{{2\text{ by 1}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 \\ 6 \end{array}} \right]}}\,=\underset{{\text{2 by 1}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {1\times 5+2\times 6} \\ {3\times 5+4\times 6} \end{array}} \right]}}\,=\underset{{\text{2 by 1}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {17} \\ {39} \end{array}} \right]}}\,

Python
m=[[1 2]
 [3 4]]
v=[5 6]
dot product mv=[17 39]
dot product vm=[23 34]

Recall that when a scalar is used to multiply a vector or a matrix, that the scalar is automatically resized to match the dimensions of the array it is being multiplied by. The same thing happens when a vector is multiplied by a matrix, the vector is resized to suit the additional dimension of the matrix.

This works fine with a regular vector unless there is a square matrix, here conversely it is the multiply function which needs to have the vector specified explicitly as either a row or a column.

\displaystyle \underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,*\underset{{1\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & 6 \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,*\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & 6 \\ 5 & 6 \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {1\times 5} & {2\times 6} \\ {3\times 5} & {4\times 6} \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & {12} \\ {15} & {24} \end{array}} \right]}}\,

\displaystyle \underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,*\underset{{2\text{ by 1}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 \\ 6 \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,*\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & 5 \\ 6 & 6 \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {1\times 5} & {2\times 5} \\ {3\times 6} & {4\times 6} \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 5 & {10} \\ {18} & {24} \end{array}} \right]}}\,

Python
m=[[1 2]
 [3 4]]
v=[[5]
 [6]]
multiply mv=[[ 5 10]
 [18 24]]
v=[[5 6]]
multiply mv=[[ 5 12]
 [15 24]]

Element by Element Exponentiation

Element by element exponentiation is the element by element multiplication of an array by itself for a specified number of times:

\displaystyle \begin{array}{l}\\{{\underset{{2\text{ by 3}}}{\mathop {\left[ {\begin{array}{*{20}{r}} 1 & 2 & 3 \\ 4 & 5 & 6 \end{array}} \right]}}\,}^{2}}=\underset{{2\text{ by 3}}}{\mathop {\left[ {\begin{array}{*{20}{r}} 1 & 2 & 3 \\ 4 & 5 & 6 \end{array}} \right]}}\,*\underset{{2\text{ by 3}}}{\mathop {\left[ {\begin{array}{*{20}{r}} 1 & 2 & 3 \\ 4 & 5 & 6 \end{array}} \right]}}\,=\underset{{2\text{ by 3}}}{\mathop {\left[ {\begin{array}{*{20}{r}} {{{1}^{2}}} & {{{2}^{2}}} & {{{3}^{2}}} \\ {{{4}^{2}}} & {{{5}^{2}}} & {{{6}^{2}}} \end{array}} \right]}}\,=\underset{{2\text{ by 3}}}{\mathop {\left[ {\begin{array}{*{20}{r}} 1 & 4 & 9 \\ {16} & {25} & {36} \end{array}} \right]}}\,\end{array}

It can be carried out in Python using the function m**p or np.power(m,p).

Python
m=[[1 2 3]
 [4 5 6]]
multiply mm=[[ 1  4  9]
 [16 25 36]]
multiply mm=[[ 1  4  9]
 [16 25 36]]
power mm=[[ 1  4  9]
 [16 25 36]]
power mm=[[ 1  4  9]
 [16 25 36]]

Array Exponentiation of Square Matrices

Array Exponentiation is far less common. Recall that in order to carry out the dot product the inner dimensions of the two matrices being multiplied must match. The only case where the inner dimension of a matrix matches the outer dimension is when the matrix is square. Matrix exponentiation can only be carried out using square matrices for example:

If we take an array exponentiated to 2, then it essentially means that we calculate the dot product of a matrix by itself.

The matrix power is found within the numpy linear algebra library, there is no shortcut like *, @ or ** as it isn’t as commonly used.

\displaystyle \underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,\underset{{2\text{ by 2}}}{\mathop {\text{@}\left[ {\begin{array}{*{20}{c}} 1 & 2 \\ 3 & 4 \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} {1\times 1+2\times 3} & {1\times 2+2\times 4} \\ {3\times 1+4\times 3} & {3\times 2+4\times 4} \end{array}} \right]}}\,=\underset{{2\text{ by 2}}}{\mathop {\left[ {\begin{array}{*{20}{c}} 7 & {10} \\ {15} & {22} \end{array}} \right]}}\,

Python
m=[[1 2]
 [3 4]]
dot product mm=[[ 7 10]
 [15 22]]
dot product mm=[[ 7 10]
 [15 22]]
Advertisements

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.