Vector Spaces
Merriam-webster dictionary defines a vector as
“….a quantity that has magnitude and direction and that is commonly represented by a directed line segment whose length represents the magnitude and whose orientation in space represents the direction…”
A vector is an “arrow” that goes from an origin point to a feature of space.
According to this definition, each vector v has the following properties:
Norm of a vector
Each vector has a length, or norm, associated with it. Norm of a vector \(\vec{v}\) is denoted by \(|\vec{v}|\). Vectors with unit length are called “normal”.
Angle between two vectors
The angle between two vectors \(\vec{v_1}\) and \(\vec{v_2}\) is denoted by \(\measuredangle \vec{v_1} \vec{v_2}\). Angle is directed: \(\measuredangle \vec{v_1} \vec{v_2} = -\measuredangle \vec{v_2} \vec{v_1}\).
There are two basic operations defined in a vector space
- Multiplication by a scalar number: Given a vector v, we can define 5v, -3v, 2v etc. In each case, the length of the vector changes but its orientation (ie, the angles it makes with other vectors) dont change
- Addition: Given two vectors v and w, we can add them as v+w and get a new vector. Addition is done by the parallelogram rule. Most probably you have added two given vectors in high school with the aid of a ruler and a compass.
.These operations must satisfy the axioms for addition and scalar multiplication. As these axioms are well known, we will not list them here.
Multiplication of two vectors
Addition of two vectors and multiplication of a vector with a scalar are defined by means of the axioms of vector space. A more knotty problem is to multiply two vectors. Two different vector multiplications exist: Dot and cross products. For our purposes, only dot products are relevant.
Dot Product
Dot product of two vectors give a scalar. It is defined by
\begin{eqnarray}
\vec{v}.\vec{w}=|\vec{v}||\vec{w}|\cos(\theta)
\end{eqnarray}
where \(\theta=\measuredangle \vec{v_1} \vec{v_2}\) is the angle between the two vectors.
Note that dot product of a vector with itself is
\begin{eqnarray}
\vec{v}.\vec{v}=|\vec{v}||\vec{v}|\cos(0)=|\vec{v}|^2
\end{eqnarray}
hence norm can be redefined as
\begin{eqnarray}
|\vec{v}|=\sqrt{\vec{v}.\vec{v}}
\end{eqnarray}
Vector Bases in three dimensional spaces
In three dimensional space, we can pick three arbitrary vectors and express all the other vectors as a linear combination of these three vectors. The only restriction on these three vectors is that they should be linearly independent.
Let us pick three linearly independent vectors \(\vec{e_1}, \vec{e_2}, \vec{e_3} \). Then any other vector \(\vec{w}\) of the space can be written as
\begin{eqnarray}
\vec{w}=\alpha_1 \vec{e_1} + \alpha_2 \vec{e_2} + \alpha_3 \vec{e_3}
\end{eqnarray}
The set \(E=\{\vec{e_1}, \vec{e_2}, \vec{e_3}\} \) is called a basis and the scalars \(\alpha_1,\alpha_2,\alpha_3\) are called “the components of vector \(\vec{w}\) in basis \(B\)”. An equivalent notation for (xx) is
\begin{eqnarray}
\vec{w} =\begin{bmatrix}
\alpha_1\\
\alpha_2\\
\alpha_3
\end{bmatrix}_E
\end{eqnarray}
This means that given a basis, each vector can be represented by three numbers.
Note that we can pick a completely different set of basis vectors as \(U=\{\vec{u_1}, \vec{u_2}, \vec{u_3}\} \). In this new basis, our old vector \(\vec{w}\) will be expressed as
\begin{eqnarray}
\vec{w}=\beta_1 \vec{u_1} + \beta_2 \vec{u_2} + \beta_3 \vec{u_3}
\end{eqnarray}
In this new basis, it will have a completely different representation:
\begin{eqnarray}
\vec{w} =\begin{bmatrix}
\beta_1\\
\beta_2\\
\beta_3
\end{bmatrix}_U
\end{eqnarray}
and these two representations are equivalent.
\begin{eqnarray}
\vec{w} =\begin{bmatrix}
\alpha_1\\
\alpha_2\\
\alpha_3
\end{bmatrix}_E = \begin{bmatrix}
\beta_1\\
\beta_2\\
\beta_3
\end{bmatrix}_U
\end{eqnarray}
Naturally, the basis vectors are expressed in their own basis as
\begin{eqnarray}
\vec{e_1} =\begin{bmatrix}
1\\
0\\
0
\end{bmatrix}_E, \vec{e_2} = \begin{bmatrix}
0\\
1\\
0
\end{bmatrix}_E, \vec{e_3} = \begin{bmatrix}
0\\
0\\
1
\end{bmatrix}_E
\end{eqnarray}
Dot product with orthonormal Basis Vectors
In orthonormal case, the basis vectors are orthonormal, ie, they are of unit length (normal)
\begin{eqnarray}
|\vec{e_i}|=1
\end{eqnarray}
and othogonal to each other
\begin{eqnarray}
\vec{e_i}.\vec{e_j}=\delta_{ij}
\end{eqnarray}
The dot product definition can be specified to basis. If we define the vectors \(\vec{v}\), \(\vec{w}\) in the basis $B$ as
\begin{eqnarray}
\vec{v}=a_1 \vec{e_1} +a_2 \vec{e_2} +a_3 \vec{e_3} \\
\vec{w}=b_1 \vec{e_1} +b_2 \vec{e_2} +b_3 \vec{e_3}
\end{eqnarray}
Then the dot product of them is
\begin{eqnarray}
\vec{v}. \vec{w} &=& (a_1 \vec{e_1} +a_2 \vec{e_2} +a_3 \vec{e_3}).(b_1 \vec{e_1} +b_2 \vec{e_2} +b_3 \vec{e_3})\\
&=& a_1b_1 \vec{e_1}. \vec{e_1}+ a_1b_2 \vec{e_1}. \vec{e_2}+ a_1b_3 \vec{e_1}. \vec{e_3}+ \\
&& a_2b_1 \vec{e_2}. \vec{e_1}+ a_2b_2 \vec{e_2}. \vec{e_2}+ a_2b_3 \vec{e_2}. \vec{e_3}+ \\
&& a_3b_1 \vec{e_3}. \vec{e_1}+ a_3b_2 \vec{e_3}. \vec{e_2}+ a_3b_3 \vec{e_3}. \vec{e_3}
\end{eqnarray}
using equations .., this simplifies to
\begin{eqnarray}
\vec{v}. \vec{w}=a_1 b_1 + a_2 b_2 + a_3 b_3
\end{eqnarray}
Or, in tupple notation
\begin{eqnarray}
\vec{v}. \vec{w}= \vec{v}^T \vec{w}= \begin{bmatrix}
a_1
a_2
a_3
\end{bmatrix} \begin{bmatrix}
b_1\\
b_2\\
b_3
\end{bmatrix} =a_1 b_1 + a_2 b_2 + a_3 b_3
\end{eqnarray}
Please note carefully that this formula for inner products is only valid in orthonormal bases. So, do not use it in non-orthonormal bases. The fully general formula will be given in next subsection.
The norm in orthonormal basis can be expressed as
\begin{eqnarray}
|\vec{v}|=\sqrt{\vec{v}.\vec{v}}=\sqrt{a_1^2 + a_2^2 + a_3^2}
\end{eqnarray}
The angle \(\theta\) between \(\vec{v}\) and \(\vec{v}\) can be found from (..)
\begin{eqnarray}
\theta &=& \arctan \left( \frac{\vec{v}.\vec{w}}{|\vec{v}||\vec{w}|} \right) \\
&=& \arctan \left( \frac{a_1 b_1 + a_2 b_2 + a_3 b_3}{\sqrt{a_1^2 + a_2^2 + a_3^2} \sqrt{b_1^2 + b_2^2 + b_3^2 } } \right)
\end{eqnarray}
Dot product with Non-Orthonormal Basis vectors
Nonorthonormal bases will not concern us in this lecture. But we just want to show how complicated the dot product formulas become in the nonorthonormla case, and what a great blessing the orthonormal case is.
Let us assume that we want to use three non-orthonormal vectors \(\vec{u_1}, \vec{u_2}, \vec{u_3}\) as a basis. This means that we have
\begin{eqnarray}
\vec{u_i}.\vec{u_j}=g_{ij}
\end{eqnarray}
instead of (..), which makes things much more complicated.
If we define the vectors \(\vec{v}\), \(\vec{v}\) in the basis \(B\) as
\begin{eqnarray}
\vec{v}=a_1 \vec{u_1} +a_2 \vec{u_2} +a_3 \vec{u_3} \\
\vec{w}=b_1 \vec{u_1} +b_2 \vec{u_2} +b_3 \vec{u_3}
\end{eqnarray}
Then the dot product of them is
\begin{eqnarray}
\vec{v}. \vec{w} &=& a_1b_1 \vec{u_1}. \vec{u_1}+ a_1b_2 \vec{u_1}. \vec{u_2}+ a_1b_3 \vec{u_1}. \vec{u_3}+ \\
&& a_2b_1 \vec{u_2}. \vec{u_1}+ a_2b_2 \vec{u_2}. \vec{u_2}+ a_2b_3 \vec{u_2}. \vec{u_3}+ \\
&& a_3b_1 \vec{u_3}. \vec{u_1}+ a_3b_2 \vec{u_3}. \vec{u_2}+ a_3b_3 \vec{u_3}. \vec{u_3}\\ \\
&=& g_{11} a_1b_1 + g_{12} a_1b_2 + g_{13} a_1b_3 + \\
&& g_{21} a_2b_1 + g_{22} a_2b_2 + g_{23} a_2b_3 + \\
&& g_{31} a_3b_1 + g_{32} a_3b_2 + g_{33} a_3b_3 \\\\
&=& \sum_{i=1}^3 \sum_{j=1}^3 g_{ij} a_i b_j
\end{eqnarray}
and the norm equation becomes
\begin{eqnarray}
|\vec{v}|=\sqrt{\vec{v}.\vec{v}}=\sqrt{\sum_{i=1}^3 \sum_{j=1}^3 g_{ij} a_i a_j}
\end{eqnarray}
and the angle between the two vectors $\vec{v}$, $\vec{v}$ becomes
\begin{eqnarray}
\theta &=& \arctan \left( \frac{\vec{v}.\vec{w}}{|\vec{v}||\vec{w}|} \right) \\
&=& \arctan \left( \frac{\sum_{i=1}^3 \sum_{j=1}^3 g_{ij} a_i b_j}{\sqrt{\sum_{i=1}^3 \sum_{j=1}^3 g_{ij} a_i a_j} \sqrt{\sum_{i=1}^3 \sum_{j=1}^3 g_{ij} b_i b_j } } \right)
\end{eqnarray}
As can be seen, the non-orthonormal formulas are considerably more complicated then their orthonormal versions.
Expressing a vector in an Orthonormal basis
Let us trry to find the representation of the vector \(\vec{v}\) in the orthonormal basis \(\vec{e_1}, \vec{e_2}, \vec{e_3} \). This representation must be of the form
\begin{eqnarray}
\vec{v}=a_1 \vec{e_1} +a_2 \vec{e_2} +a_3 \vec{e_3}
\end{eqnarray}
when we finf the numbers \(a_1, a_2, a_3\) we can say that we have successfully found the representation.
Let us take the dot product of both sides with \(\vec{e_1}\). Using orthonormality properties
\begin{eqnarray}
\vec{v}.\vec{e_1}=(a_1 \vec{e_1} +a_2 \vec{e_2} +a_3 \vec{e_3}).\vec{e_1} = a_1
\end{eqnarray}
Generalizing, we can write
\begin{eqnarray}
a_1&=&\vec{v}.\vec{e_1} \\
a_2&=&\vec{v}.\vec{e_2} \\
a_3&=&\vec{v}.\vec{e_3}
\end{eqnarray}
Hence,
\begin{eqnarray}
\vec{v}=(\vec{v}.\vec{e_1}) \vec{e_1} +(\vec{v}.\vec{e_2}) \vec{e_2} +(\vec{v}.\vec{e_3}) \vec{e_3}
\end{eqnarray}
Expressing a vector in a non-Orthonormal basis
Let us trry to find the representation of the vector \(\vec{v}\) in the non-orthonormal basis \(\vec{u_1}, \vec{u_2}, \vec{u_3} \). This representation must still be of the form
\begin{eqnarray}
\vec{v}=b_1 \vec{u_1} +b_2 \vec{u_2} +b_3 \vec{u_3}
\end{eqnarray}
and the problem is to find $b_1$’s. We dont have the orthonormality relation between the basis vectors. Instead, we have
\begin{eqnarray}
\vec{u_i}.\vec{u_j}=g_{ij}
\end{eqnarray}
Multiplying both sides of the equation by \(\vec{u_1}, \vec{u_2}, \vec{u_3} \)
\begin{eqnarray}
\vec{v}.\vec{u_1}=(b_1 \vec{u_1} +b_2 \vec{u_2} +b_3 \vec{u_3}).\vec{u_1}\\
\vec{v}.\vec{u_2}=(b_1 \vec{u_1} +b_2 \vec{u_2} +b_3 \vec{u_3}).\vec{u_2}\\
\vec{v}.\vec{u_3}=(b_1 \vec{u_1} +b_2 \vec{u_2} +b_3 \vec{u_3}).\vec{u_3}
\end{eqnarray}
or
\begin{eqnarray}
\vec{v}.\vec{u_1}=b_1 \vec{u_1}.\vec{u_1} +b_2 \vec{u_2}.\vec{u_1} +b_3 \vec{u_3}.\vec{u_1}\\
\vec{v}.\vec{u_2}=b_1 \vec{u_1}.\vec{u_2} +b_2 \vec{u_2}.\vec{u_2} +b_3 \vec{u_3}.\vec{u_2}\\
\vec{v}.\vec{u_3}=b_1 \vec{u_1}.\vec{u_3} +b_2 \vec{u_2}.\vec{u_3} +b_3 \vec{e_3}.\vec{u_3}
\end{eqnarray}
which is three equations in three unknowns, that can be written in matrix form:
\begin{eqnarray}
\begin{bmatrix}
\vec{v}.\vec{u_1}\\
\vec{v}.\vec{u_1}\\
\vec{v}.\vec{u_1}
\end{bmatrix} &=& \begin{bmatrix}
\vec{u_1}.\vec{u_1} & \vec{u_1}.\vec{u_2} & \vec{u_1}.\vec{u_3} \\
\vec{u_1}.\vec{u_2} & \vec{u_2}.\vec{u_2} & \vec{u_2}.\vec{u_3} \\
\vec{u_1}.\vec{u_3} & \vec{u_2}.\vec{u_3} & \vec{u_3}.\vec{u_3}
\end{bmatrix}
\begin{bmatrix}
b_1\\
b_2\\
b_3
\end{bmatrix} \\\\
&=& \begin{bmatrix}
g_{11} & g_{12} & g_{13} \\
g_{12} & g_{22} & g_{23} \\
g_{13} & g_{23} & g_{33}
\end{bmatrix}
\begin{bmatrix}
b_1\\
b_2\\
b_3
\end{bmatrix}
\end{eqnarray}
As can be seen the non-orthonormal case cannot be reduced to a nice, simple form like the orthonormal case. Solving the equations we get the unknown $b_i$’s.
\begin{eqnarray}
\begin{bmatrix}
b_1\\
b_2\\
b_3
\end{bmatrix} = \begin{bmatrix}
g_{11} & g_{12} & g_{13} \\
g_{12} & g_{22} & g_{23} \\
g_{13} & g_{23} & g_{33}
\end{bmatrix}^{-1}
\begin{bmatrix}
\vec{v}.\vec{u_1}\\
\vec{v}.\vec{u_1}\\
\vec{v}.\vec{u_1}
\end{bmatrix}
\end{eqnarray}
The matrix $G$
\begin{eqnarray}
G=\begin{bmatrix}
g_{11} & g_{12} & g_{13} \\
g_{12} & g_{22} & g_{23} \\
g_{13} & g_{23} & g_{33}
\end{bmatrix}
\end{eqnarray}
is known as the “metric tensor”. It is reduced to the identity matrix when we use an orthonormal system. This is the main reason why we use orthonormal systems.