Orthogonality, Least Squares and the Spectral Theorem: every key term you need (+ practice quiz)
25 flashcard terms for Linear Algebra Topic 8, written to match the course framework. Study them here, then drill them as interactive flashcards, or test yourself with the 15-question quiz — free, no account needed.
A rule pairing two vectors with a scalar that is linear in each slot, symmetric, and positive on nonzero vectors; on coordinate space it is the sum of products of matching entries.
Norm of a vector
The square root of the inner product of a vector with itself, measuring its length. It is zero only for the zero vector and scales by the absolute value of a scalar.
Unit vector
A vector of norm one. Dividing any nonzero vector by its norm produces a unit vector in the same direction, a step called normalization.
Distance between vectors
The norm of their difference. It satisfies the triangle inequality, so the distance from one vector to another never exceeds a detour through a third.
Orthogonal vectors
Two vectors whose inner product is zero. The zero vector is orthogonal to everything, and orthogonality generalizes the idea of a right angle.
Pythagorean identity
For orthogonal vectors the squared norm of the sum equals the sum of the squared norms; the identity fails precisely when the inner product is nonzero.
Cauchy Schwarz inequality
The absolute value of the inner product of two vectors never exceeds the product of their norms, with equality exactly when the vectors are parallel.
Orthogonal set
A collection of nonzero vectors that are pairwise orthogonal. Any such set is automatically linearly independent, so it is a basis of the subspace it spans.
Orthonormal set
An orthogonal set whose members all have norm one. Coordinates relative to such a set are computed by simple inner products, with no system to solve.
Orthogonal complement
The set of all vectors orthogonal to every member of a given subspace. It is itself a subspace, and its dimension plus the original dimension gives the whole dimension.
Fundamental subspace relation
The orthogonal complement of the row space of a matrix is its null space, and the complement of the column space is the null space of the transpose.
Orthogonal projection onto a subspace
The unique closest point of a subspace to a given vector, characterized by the property that the difference is orthogonal to every vector in the subspace.
Best approximation theorem
The orthogonal projection of a vector onto a subspace is strictly closer to that vector than any other point of the subspace, which is what makes it the best approximation.
Orthogonal decomposition
Every vector splits uniquely as a sum of a member of a subspace and a member of its orthogonal complement, the two pieces being the projections onto each part.
Gram Schmidt process
A procedure turning any basis into an orthogonal one by subtracting from each vector its projections onto the previously produced vectors, preserving the span at every stage.
QR factorization
A writing of a matrix with independent columns as a matrix with orthonormal columns times an upper triangular matrix with positive diagonal entries.
Orthogonal matrix
A square matrix whose columns are orthonormal, so its transpose is its inverse. Such matrices preserve norms, inner products and therefore angles.
Least squares problem
The task of finding a vector minimizing the distance from a target to the column space when a system is inconsistent, giving the closest possible approximate solution.
Normal equations
The square system obtained by multiplying both sides of the original system on the left by the transpose of the coefficient matrix; its solutions are exactly the least squares solutions.
Uniqueness of the least squares solution
The least squares solution is unique exactly when the columns of the coefficient matrix are independent, in which case the normal equations have an invertible matrix.
Residual vector
The difference between the observed target and the projection onto the column space. Its norm measures how badly the original system fails to be consistent.
Regression line by least squares
The straight line minimizing the sum of squared vertical distances from a set of data points, obtained by solving the normal equations for slope and intercept.
Spectral theorem for symmetric matrices
A real symmetric matrix has only real eigenvalues, eigenspaces for distinct eigenvalues are mutually orthogonal, and an orthonormal basis of eigenvectors always exists.
Orthogonal diagonalization
A factorization of a matrix as an orthogonal matrix times a diagonal matrix times the transpose of the orthogonal matrix. It is possible exactly for real symmetric matrices.
Quadratic form
A function assigning to each vector the value of an expression built from a symmetric matrix; orthogonal diagonalization removes cross terms and reveals whether it is positive definite.