These examples will illustrate key concepts and provide insights into the mathematical computations involved in each area in Linear Algebra.
Linear Algebra Topics – Decoded with Examples and Calculations
1. Linear Equations
A linear equation is an equation involving a linear combination of variables. The standard form for one linear equation with \(n\) variables is:
\[
a_1 x_1 + a_2 x_2 + \dots + a_n x_n = b,
\]
where \(a_1, a_2, \dots, a_n\) are constants and \(x_1, x_2, \dots, x_n\) are variables.
Example: Solve the system of equations:
\[
\begin{aligned}
x + 2y – z &= 3, \\
2x – y + 3z &= 7, \\
3x + y + 2z &= 10.
\end{aligned}
\]
Use Gaussian elimination to solve the system. We convert this system to its augmented matrix form:
\[
\begin{pmatrix}
1 & 2 & -1 & 3 \\
2 & -1 & 3 & 7 \\
3 & 1 & 2 & 10
\end{pmatrix}
\]
After performing row operations to get it into row echelon form and solving, we find:
\[
x = 2, \quad y = 1, \quad z = 0.
\]
2. Vector Spaces
A vector space is a set of vectors where addition and scalar multiplication are defined and satisfy certain properties (closure under addition, existence of an additive identity, etc.).
Example: Consider the set \(V = \{ (x, y) \in \mathbb{R}^2 : x + y = 0 \}\). Is this a subspace of \(\mathbb{R}^2\)?
1. Zero vector: The zero vector \((0, 0)\) is in \(V\) because \(0 + 0 = 0\).
2. Closure under addition: If \(u = (x_1, y_1)\) and \(v = (x_2, y_2)\) are in \(V\), then \(x_1 + y_1 = 0\) and \(x_2 + y_2 = 0\). Their sum is \(u + v = (x_1 + x_2, y_1 + y_2)\), and \( (x_1 + x_2) + (y_1 + y_2) = 0 \), so \(u + v \in V\).
3. Closure under scalar multiplication: If \(c\) is a scalar and \(u = (x, y)\) is in \(V\), then \(c(x + y) = c(0) = 0\), so \(c \cdot u \in V\).
Therefore, \(V\) is a subspace.
3. Linear Transformations
A linear transformation \(T: V \to W\) between two vector spaces \(V\) and \(W\) is a function that satisfies:
1. \(T(u + v) = T(u) + T(v)\) for all \(u, v \in V\),
2. \(T(c \cdot u) = c \cdot T(u)\) for all scalars \(c\).
Example: Let \(T: \mathbb{R}^2 \to \mathbb{R}^2\) be defined by \(T(x, y) = (x + 2y, 3x – y)\).
To verify if this is a linear transformation, check the two properties:
1. Additivity: For vectors \(u = (x_1, y_1)\) and \(v = (x_2, y_2)\),
\[
T(u + v) = T((x_1 + x_2, y_1 + y_2)) = (x_1 + x_2 + 2(y_1 + y_2), 3(x_1 + x_2) – (y_1 + y_2)),
\]
which simplifies to:
\[
T(u + v) = (x_1 + 2y_1 + x_2 + 2y_2, 3x_1 + 3x_2 – y_1 – y_2) = T(u) + T(v).
\]
2. Homogeneity: For scalar \(c\) and vector \(u = (x, y)\),
\[
T(c \cdot u) = T(c \cdot (x, y)) = (c \cdot x + 2(c \cdot y), 3(c \cdot x) – c \cdot y) = c \cdot T(u).
\]
Thus, \(T\) is a linear transformation.
4. Eigenvalues and Eigenvectors
An eigenvector of a matrix \(A\) is a nonzero vector \(v\) such that:
\[
A \cdot v = \lambda \cdot v,
\]
where \(\lambda\) is the corresponding eigenvalue.
Example: Find the eigenvalues and eigenvectors of:
\[
A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}.
\]
1. Find the characteristic equation:
\[
\text{det}(A – \lambda I) = \text{det}\begin{pmatrix} 4 – \lambda & 1 \\ 2 & 3 – \lambda \end{pmatrix} = (4 – \lambda)(3 – \lambda) – 2.
\]
Simplifying:
\[
\lambda^2 – 7\lambda + 10 = 0.
\]
The eigenvalues are \(\lambda_1 = 5\) and \(\lambda_2 = 2\).
2. Find the eigenvectors for \(\lambda_1 = 5\):
Solve:
\[
\begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = 0.
\]
This gives \(v_1 = v_2\), so the eigenvector is \(v = \begin{pmatrix} 1 \\ 1 \end{pmatrix}\).
5. Matrices
A matrix is a rectangular array of numbers, and operations such as addition, multiplication, and finding the determinant can be performed on matrices.
Example: Compute the product of two matrices:
\[
A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}.
\]
The matrix product \(A \cdot B\) is:
\[
A \cdot B = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} \cdot \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix} = \begin{pmatrix} 1\cdot5 + 2\cdot7 & 1\cdot6 + 2\cdot8 \\ 3\cdot5 + 4\cdot7 & 3\cdot6 + 4\cdot8 \end{pmatrix} = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}.
\]
6. Matrix Inverses
The inverse of a matrix \(A\) is denoted \(A^{-1}\) and satisfies:
\[
A \cdot A^{-1} = A^{-1} \cdot A = I,
\]
where \(I\) is the identity matrix.
Example: Find the inverse of:
\[
A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}.
\]
1. Compute the determinant of \(A\):
\[
\text{det}(A) = (1 \times 4) – (2 \times 3) = 4 – 6 = -2.
\]
2. Use the formula for the inverse:
\[
A^{-1} = \frac{1}{\text{det}(A)} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix} = \frac{1}{-2} \begin{pmatrix} 4 & -2 \\ -3 & 1 \end{pmatrix} = \begin{pmatrix} -2 & 1 \\ 1.5 & -0.5 \end{pmatrix}.
\]
7. Determinants
The determinant of a square matrix is a scalar value that can be computed from its elements. It is
used to determine if a matrix is invertible and to find eigenvalues.
Example: Calculate the determinant of:
\[
A = \begin{pmatrix} 2 & 3 \\ 5 & 7 \end{pmatrix}.
\]
The determinant is:
\[
\text{det}(A) = (2 \times 7) – (3 \times 5) = 14 – 15 = -1.
\]
8. Inner Product Spaces
An inner product space is a vector space with an additional structure called the inner product. The inner product of two vectors \(u\) and \(v\) in \(\mathbb{R}^n\) is defined as:
\[
\langle u, v \rangle = u_1v_1 + u_2v_2 + \dots + u_nv_n.
\]
Example: Compute the inner product of \(u = (1, 2)\) and \(v = (3, 4)\):
\[
\langle u, v \rangle = (1)(3) + (2)(4) = 3 + 8 = 11.
\]
9. Linear Independence
Vectors \(v_1, v_2, \dots, v_n\) are said to be **linearly independent** if no nontrivial linear combination of them equals the zero vector.
Example: Check if the vectors \(v_1 = (1, 2)\) and \(v_2 = (2, 4)\) are linearly independent.
To check linear independence, solve:
\[
c_1 v_1 + c_2 v_2 = 0.
\]
This gives the system:
\[
c_1(1, 2) + c_2(2, 4) = (0, 0).
\]
Solving this system, we find \(c_1 = 0\) and \(c_2 = 0\). Since the only solution is the trivial solution, \(v_1\) and \(v_2\) are linearly dependent.
10. Matrix Congruence
Matrices \(A\) and \(B\) are said to be congruent if there exists an invertible matrix \(P\) such that:
\[
A = P^T B P.
\]
11. Vectors
A vector is an element of a vector space. It has both magnitude and direction.
Example: Let \(u = (1, 2)\) and \(v = (3, 4)\). The sum of the vectors is:
\[
u + v = (1 + 3, 2 + 4) = (4, 6).
\]
The scalar multiplication of \(u\) by 2 is:
\[
2 \cdot u = (2 \cdot 1, 2 \cdot 2) = (2, 4).
\]
12. Euclidean Vector Spaces
The Euclidean space \(\mathbb{R}^n\) is a space of \(n\)-dimensional vectors with the standard dot product as the inner product.
Example: Find the dot product of vectors \(u = (1, 2, 3)\) and \(v = (4, -5, 6)\):
\[
\langle u, v \rangle = (1)(4) + (2)(-5) + (3)(6) = 4 – 10 + 18 = 12.
\]
13. Gram-Schmidt Process
The Gram-Schmidt process is a method for orthonormalizing a set of vectors in an inner product space.
Example: Given two linearly independent vectors \(v_1 = (1, 0)\) and \(v_2 = (1, 1)\), use the Gram-Schmidt process to construct an orthonormal set:
1. \(u_1 = v_1 = (1, 0)\).
2. Project \(v_2\) onto \(u_1\):
\[
\text{proj}_{u_1}v_2 = \frac{\langle v_2, u_1 \rangle}{\langle u_1, u_1 \rangle} u_1 = \frac{1}{1}(1, 0) = (1, 0).
\]
3. Subtract the projection from \(v_2\) to get \(u_2\):
\[
u_2 = v_2 – \text{proj}_{u_1} v_2 = (1, 1) – (1, 0) = (0, 1).
\]
4. Normalize \(u_1\) and \(u_2\) to get the orthonormal vectors:
\[
e_1 = \frac{u_1}{\|u_1\|} = (1, 0), \quad e_2 = \frac{u_2}{\|u_2\|} = (0, 1).
\]
Thus, the orthonormal set is \(\{(1, 0), (0, 1)\}\).
14. Orthogonal Matrices
A matrix \(Q\) is orthogonal if its rows and columns are orthonormal vectors, i.e., \(Q^T Q = Q Q^T = I\).
Example: The matrix
\[
Q = \begin{pmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{pmatrix}
\]
is orthogonal because:
\[
Q^T Q = \begin{pmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end{pmatrix} \begin{pmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{pmatrix} = I.
\]
15. Singular Value Decomposition (SVD)
The Singular Value Decomposition (SVD) of a matrix \(A\) is a factorization of \(A\) as:
\[
A = U \Sigma V^T,
\]
where \(U\) and \(V\) are orthogonal matrices and \(\Sigma\) is a diagonal matrix containing the singular values of \(A\).
16. Solving Systems of Equations with Matrices
To solve a system of linear equations \(Ax = b\), we can use matrix inversion or Gaussian elimination.
17. Coordinates
Coordinates represent the position of a point in a vector space with respect to a given basis.
18. Dimension and Subspaces
The dimension of a vector space is the number of vectors in any basis of the space. A subspace is a subset of a vector space that is itself a vector space.
19. Matrix Exponential
The matrix exponential is defined as:
\[
e^A = I + A + \frac{A^2}{2!} + \frac{A^3}{3!} + \dots
\]
It is used in differential equations and systems theory.
20. Orthogonal Projections
An orthogonal projection of a vector \(v\) onto a subspace \(W\) is the closest vector in \(W\) to \(v\).
21. Projections
A projection is a linear transformation that maps vectors onto a subspace.
22. Subspaces
A subspace is a subset of a vector space that is itself a vector space under the same operations as the original space.
23. Addition, Subtraction, and Scalar Multiplications
These are the basic operations in a vector space, where addition and scalar multiplication follow specific rules.
24. Computations
These involve performing operations such as matrix multiplication, finding eigenvalues, and solving systems of equations using techniques like Gaussian elimination.
