The SVD-Fundamental Theorem of Linear Algebra

Abstract

Given an m × n matrix A, with m ≥ n, the four subspaces associated with it are shown in Fig. 1 (see [1]).

Fig. 1. The row spaces and the nullspaces of A and A^T; a₁ through a_n and h₁ through h_m are abbreviations of the alignerframe and hangerframe vectors respectively (see [2]).

The Fundamental Theorem of Linear Algebra tells us that N(A) is the orthogonal complement of R(A^T). These four subspaces tell the whole story of the Linear System Ax = y.  So, for example, the absence of N(A^T) indicates that a solution always exists, whereas the absence of N(A) indicates that this solution is unique. Given the importance of these subspaces, computing bases for them is the gist of Linear Algebra. In “Classical” Linear Algebra, bases for these subspaces are computed using Gaussian Elimination; they are orthonormalized with the help of the Gram-Schmidt method. Continuing our previous work [3] and following Uhl’s excellent approach [2] we use SVD analysis to compute orthonormal bases for the four subspaces associated with A, and give a 3D explanation. We then state and prove what we call the “SVD-Fundamental Theorem” of Linear Algebra, and apply it in solving systems of linear equations.