Download + v

Review on Linear Algebra
Spring 2017
Instructor: Tai-Yue (Jason) Wang
Department of Industrial and Information Management
Institute of Information Management
Contents

Introduction to System of Linear Equations,
Matrices, and Matrix Operations

Euclidean Vector Spaces
General Vector Spaces
Inner Product Spaces
Eigenvalue and Eigenvector



2
Introduction to System of
Linear Equations, Matrices,
and Matrix Operations
Linear Equations



Any straight line in xy-plane can be represented
algebraically by an equation of the form:
a1x + a2y = b
General form: Define a linear equation in the n
variables x1, x2, …, xn :
a1x1 + a2x2 + ··· + anxn = b
where a1, a2, …, an and b are real constants.
The variables in a linear equation are sometimes
called unknowns.
4
Example (Linear Equations)

The equations
are linear



1
x  3 y  7, y  x  3 z  1, and x1  2 x2  3x3  x4  7
2
A linear equation does not involve any products or
roots of variables
All variables occur only to the first power and do not
appear as arguments for trigonometric, logarithmic,
or exponential functions.
The equations x  3 y  5, 3x  2 y  z  xz  4, and y  sin x
are not linear (non-linear)
5
Example (Linear Equations)


A solution of a linear equation is a sequence of n
numbers s1, s2, …, sn such that the equation is
satisfied.
The set of all solutions of the equation is called its
solution set or general solution of the equation.
6
a11x1  a12 x2  ...  a1n xn  b1
a21x1  a22 x2  ...  a2 n xn  b2
Linear Systems









am1 x1  am 2 x2  ...  amn xn  bm
A finite set of linear equations in the variables x1,
x2, …, xn is called a system of linear equations or
a linear system.
A sequence of numbers s1, s2, …, sn is called a
solution of the system
A system has no solution is said to be inconsistent.
If there is at least one solution of the system, it is
called consistent.
Every system of linear equations has either no
solutions, exactly one solution, or infinitely many
7
solutions
Augmented Matrices



The location of the +s, the xs, and the =s can
be abbreviated by writing only the rectangular
array of numbers.
This is called the augmented matrix (擴增矩陣)
for the system.
It must be written in the same order in each
equation as the unknowns and the constants
must be on the right
8
Augmented Matrices

In computer science
an array is a data
structure consisting of
a group of elements
that are accessed by
indexing.
It must be written in the same order in each
equation as the unknowns and the constants
must be on the right
1st column
a11x1  a12 x2  ...  a1n xn  b1
a21 x1  a22 x2  ...  a2 n xn  b2




am1 x1  am 2 x2  ...  amn xn  bm
a11 a12 ... a1n
a a ... a
2n
 21 22
  


am1 am 2 ... amn
Matrix
b1 
b2 
 

bm 
1st row
9
Homogeneous(齊次) Linear
Systems

A system of linear equations is said to be
homogeneous if the constant terms are all
zero; that is, the system has the form:
a11x1  a12 x2  ...  a1n xn  0
a21x1  a22 x2  ...  a2 n xn  0




am1 x1  am 2 x2  ...  amn xn  0
10
Homogeneous Linear Systems

Every homogeneous system of linear equation is
consistent, since all such system have x1 = 0, x2
= 0, …, xn = 0 as a solution.



This solution is called the trivial solution(零解).
If there are another solutions, they are called nontrivial
solutions(非零解).
There are only two possibilities for its solutions:


There is only the trivial solution
There are infinitely many solutions in addition to the trivial
solution
11
Theorem

Theorem 1


A homogeneous system of linear equations with more
unknowns than equations has infinitely many solutions.
Remark



This theorem applies only to homogeneous system!
A nonhomogeneous system with more unknowns than
equations need not be consistent; however, if the system
is consistent, it will have infinitely many solutions.
e.g., two parallel planes in 3-space
12
Definition and Notation


A matrix is a rectangular array of numbers. The numbers in
the array are called the entries in the matrix
A general mn matrix A is denoted as
a11 a12 ...
a
a22 ...
21

A
 


am1 am 2 ...
a1n 
a2 n 




amn 
13
Definition and Notation



The entry that occurs in row i and column j of matrix A will
be denoted aij or Aij. If aij is real number, it is common to
be referred as scalars
The preceding matrix can be written as [aij]mn or [aij]
A matrix A with n rows and n columns is called a square
matrix of order n
14
Definition


Two matrices are defined to be equal if they have the
same size and their corresponding entries are equal
 If A = [aij] and B = [bij] have the same size, then A
= B if and only if aij = bij for all i and j
If A and B are matrices of the same size, then the sum
A + B is the matrix obtained by adding the entries of
B to the corresponding entries of A.
15
Definition


The difference A – B is the matrix obtained by
subtracting the entries of B from the corresponding
entries of A
If A is any matrix and c is any scalar, then the product
cA is the matrix obtained by multiplying each entry of
the matrix A by c. The matrix cA is said to be the scalar
multiple of A
 If A = [aij], then cAij = cAij = caij
16
Definitions


If A is an mr matrix and B is an rn matrix, then
the product AB is the mn matrix whose entries
are determined as follows.
To find the entry in row i and column j of AB,
single out row i from the matrix A and column j
from the matrix B. Multiply the corresponding
entries from the row and column together and then
add up the resulting products
17
Definitions

That is, (AB)mn = Amr Brn
 a11
a
 21
 
AB  
 ai1
 

am1
a12
a22

ai 2

am 2
 a1r 
 a2 r  b11 b12  b1 j  b1n 


  b21 b22  b2 j  b2 n 



 
 air   



b
b

b

b
  r1 r 2
rj
rn 


 amr 
the entry ABij in row i and column j of AB is given by
ABij = ai1b1j + ai2b2j + ai3b3j + … + airbrj
18
Partitioned Matrices

A matrix can be subdivided or partitioned into smaller
matrices by inserting horizontal and vertical rules
between selected rows and columns
19
Partitioned Matrices

For example, three possible partitions of a 34 matrix A:
 The partition of A into four
a
a
a 
a
submatrices A11, A12, A21,
A 
A
A  a
a
a
a   

A
A

and A22
a
a
a
a  
a
a
a  r 
a
 The partition of A into its row
A  a
a
a
a   r 
matrices r1, r2, and r3
a
a
a
a  r 
 The partition of A into its
a
a
a 
a
a
  c c c
A

a
a
a
column matrices c1, c2, c3,


a
a
a
a 
and c4
11
12
13
14
21
22
23
24
11
12
21
22
31
32
33
34
11
12
13
14
1
21
22
23
24
2
31
32
33
34
3
11
12
13
14
21
22
23
24
31
32
33
34
1
2
3
20
c4 
Multiplication by Columns and by
Rows

It is possible to compute a particular row or
column of a matrix product AB without
computing the entire product:
jth column matrix of AB = A[jth column matrix of B]
ith row matrix of AB = [ith row matrix of A]B
21
Multiplication by Columns and by
Rows

If a1, a2, ..., am denote the row matrices of
A and b1 ,b2, ...,bn denote the column
matrices of B, then
AB  Ab1 b 2  b n   Ab1 Ab 2  Ab n 
 a1 
 a1 B 
a 
a B 
AB   2  B   2 

  
 


a m 
a m B 
22
Matrix Products as Linear
Combinations

Let

Then

The product Ax of a matrix A with a column
matrix x is a linear combination of the column
matrices of A with the coefficients coming from the
23
matrix x
 a11 a12  a1n 
a

a

a
21
22
2
n
 and
A
 

 


am1 am 2  amn 
 x1 
x 
x   2

 
 xn 
 a11x1  a12 x2    a1n xn 
 a11 
 a12 
 a1n 
 a x  a x  a x 
a 
a 
a 
21
1
22
2
2
n
n
21
22
  x    x      x  2n 
Ax  
n

 1   2   
  



 
 
 
am1 x1  am 2 x2    amn xn 
am1 
 am 2 
amn 
Matrix Form of a Linear System



Consider any system of m linear equations in n unknowns:
a11x1  a12 x2    a1n xn  b1
a21 x1  a22 x2    a2 n xn  b2

am1 x1  am 2 x2    amn xn  bn
 a11x1  a12 x2    a1n xn   b1 
 a x  a x    a x  b 
2n n 
 21 1 22 2
 2

 


  
a
x

a
x



a
x
m
1
1
m
2
2
mn
n

 bm 
 a11 a12  a1n   x1   b1 
a
 x  b 
a

a
21
22
2
n

 2    2 
 

      

   
a
a

a
m
1
m
2
mn

  xm  bm 
Ax  b
The matrix A is called the coefficient matrix of the system
The augmented matrix of the system is given abya  a b 
11
12
1n
1
a

a

a
b
21
22
2
n
2

A b  
 




24
am1 am 2  amn bm 
Definitions

If A is any mn matrix, then the transpose of A,
denoted by AT, is defined to be the nm matrix
that results from interchanging the rows and
columns of A

That is, the first column of AT is the first row of A,
the second column of AT is the second row of A,
and so forth
25
Definitions

If A is a square matrix, then the trace of A ,
denoted by tr(A), is defined to be the sum of the
entries on the main diagonal of A. The trace of
A is undefined if A is not a square matrix.

For an nn matrix A = [aij],
n
tr ( A)   aii
i 1
26
Properties of Matrix Operations


For real numbers a and b ,we always have ab = ba,
which is called the commutative law for
multiplication. For matrices, however, AB and BA
need not be equal.
Equality can fail to hold for three reasons:



The product AB is defined but BA is undefined.
AB and BA are both defined but have different sizes.
It is possible to have AB  BA even if both AB and
27
BA are defined and have the same size.
Theorem 2
(Properties of Matrix Arithmetic)

Assuming that the sizes of the matrices are such that the
indicated operations can be performed, the following rules
of matrix arithmetic are valid:







A+B=B+A
(commutative law for addition)
A + (B + C) = (A + B) + C (associative law for addition)
A(BC) = (AB)C
(associative law for multiplication)
A(B + C) = AB + AC (left distributive law)
(B + C)A = BA + CA (right distributive law)
A(B – C) = AB – AC,
(B – C)A = BA – CA
a(B + C) = aB + aC,
a(B – C) = aB – aC
28
Theorem 2
(Properties of Matrix Arithmetic)


(a+b)C = aC + bC,
a(bC) = (ab)C,
(a-b)C = aC – bC
a(BC) = (aB)C = B(aC)
29
Zero Matrices




A matrix, all of whose entries are zero, is called a
zero matrix
A zero matrix will be denoted by 0
If it is important to emphasize the size, we shall
write 0mn for the mn zero matrix.
In keeping with our convention of using boldface
symbols for matrices with one column, we will
denote a zero matrix with one column by 0
30
Zero Matrices

Theorem 3 (Properties of Zero Matrices)
 Assuming that the sizes of the matrices are such
that the indicated operations can be
performed ,the following rules of matrix
arithmetic are valid




A+0=0+A=A
A–A=0
0 – A = -A
A0 = 0;
0A = 0
31
Identity Matrices



A square matrix with 1s on the main diagonal
and 0s off the main diagonal is called an
identity matrix and is denoted by I, or In for
the nn identity matrix
If A is an mn matrix, then AIn = A and ImA = A
An identity matrix plays the same role in matrix
arithmetic as the number 1 plays in the
numerical relationships a·1 = 1·a = a
32
Definition


If A is a square matrix, and if a matrix B of the
same size can be found such that AB = BA = I,
then A is said to be invertible and B is called an
inverse of A. If no such matrix B can be found,
then A is said to be singular.
Remark:



The inverse of A is denoted as A-1
Not every (square) matrix has an inverse
An inverse matrix has exactly one inverse
33
Theorems

Theorem 4


If B and C are both inverses of the matrix A, then B = C
Theorem 5

The matrix
a b 
A

c d 
is invertible if ad – bc  0, in which case the inverse is
given by the formula
A1 
1  d  b
ad  bc  c a 
34
Theorems

Theorem 6

If A and B are invertible matrices of the same
size ,then AB is invertible and (AB)-1 = B-1A-1
35
Definition

If A is a square matrix, then we define the
nonnegative integer powers of A to be
A0  I
An  
AA

A (n  0)



n factors

If A is invertible, then we define the
negative integer powers to be
1 1
1
A n  ( A1 ) n  
A
A

A
 
 (n  0)
n factors
36
Theorems

Theorem 7 (Laws of Exponents)


If A is a square matrix and r and s are integers,
then ArAs = Ar+s, (Ar)s = Ars
Theorem 8 (Laws of Exponents)

If A is an invertible matrix, then:



A-1 is invertible and (A-1)-1 = A
An is invertible and (An)-1 = (A-1)n for n = 0, 1, 2, …
For any nonzero scalar k, the matrix kA is invertible
and (kA)-1 = (1/k)A-1
37
Theorems

Theorem 9 (Properties of the Transpose)

If the sizes of the matrices are such that the stated
operations can be performed, then





((AT)T = A
(A + B)T = AT + BT and (A – B)T = AT – BT
(kA)T = kAT, where k is any scalar
(AB)T = BTAT
Theorem 10 (Invertibility of a Transpose)

If A is an invertible matrix, then AT is also invertible
and (AT)-1 = (A-1)T
38
Theorems

Theorem 11


Every system of linear equations has either no
solutions, exactly one solution, or in finitely
many solutions.
Theorem 12

If A is an invertible nn matrix, then for each
n1 matrix b, the system of equations Ax = b
has exactly one solution, namely, x = A-1b.
39
Example
40
Theorems

Theorem 13

Let A be a square matrix



If B is a square matrix satisfying BA = I, then B = A-1
If B is a square matrix satisfying AB = I, then B = A-1
Theorem 14

Let A and B be square matrices of the same size. If
AB is invertible, then A and B must also be
invertible.
41
Definitions


A square matrix A is mn with m = n; the (i,j)-entries
for 1  i  m form the main diagonal of A
A diagonal matrix is a square matrix all of whose
entries not on the main diagonal equal zero. By
diag(d1, …, dm) is meant the mm diagonal matrix
whose (i,i)-entry equals di for 1  i  m
42
Definitions



A mn lower-triangular matrix L satisfies (L)ij = 0 if i <
j, for 1  i  m and 1  j  n
A mn upper-triangular matrix U satisfies (U)ij = 0 if i
> j, for 1  i  m and 1  j  n
A unit-lower (or –upper)-triangular matrix T is a lower
(or upper)-triangular matrix satisfying (T)ii = 1 for 1  i
 min(m,n)
43
Properties of Diagonal Matrices

A general nn diagonal
matrix D can be written as

A diagonal matrix is
invertible if and only if all
of its diagonal entries are
nonzero
Powers of diagonal
matrices are easy to
compute

d1 0  0 
0 d  0 
2

D
 



0
0

d
n

0

0 
1 / d1
 0 1/ d 

0
2

D 1  
 

 


0
0

1
/
d
n

d1k

0
Dk  


 0
0

0


 d nk 
0 
d 2k 

0
44
Properties of Diagonal Matrices

Matrix products that involve diagonal
factors are especially easy to compute
45
Theorem


The transpose of a lower triangular matrix is
upper triangular, and the transpose of an
upper triangular matrix is lower triangular
The product of lower triangular matrices is
lower triangular, and the product of upper
triangular matrices is upper triangular
46
Theorem


A triangular matrix is invertible if and only
if its diagonal entries are all nonzero
The inverse of an invertible lower
triangular matrix is lower triangular, and
the inverse of an invertible upper triangular
matrix is upper triangular
47
Symmetric Matrices

Definition


A (square) matrix A for which AT = A, so that
Aij = Aji for all i and j, is said to be
symmetric.
Theorem 17

If A and B are symmetric matrices with the
same size, and if k is any scalar, then



AT is symmetric
A + B and A – B are symmetric
kA is symmetric
48
Symmetric Matrices

Remark

The product of two symmetric matrices is
symmetric if and only if the matrices
commute, i.e., AB = BA
49
Theorems

Theorem 18


If A is an invertible symmetric matrix, then A-1
is symmetric.
Remark:


In general, a symmetric matrix needs not be
invertible.
The products AAT and ATA are always
symmetric
50
Theorems

Theorem 19

If A is an invertible matrix, then AAT and ATA
are also invertible
51
Example
52
Euclidean Vector Spaces
Definitions

If n is a positive integer, an ordered n-tuple
(vector) is a sequence of n real numbers
(a1,a2,…,an). The set of all ordered n-tuple
is called n-space and is denoted by Rn.
54
Definitions


Two vectors u = (u1 ,u2 ,…,un) and v = (v1 ,v2 ,…,
vn) in Rn are called equal if
u1 = v1 ,u2 = v2 , …, un = vn
The sum u + v is defined by
u + v = (u1+v1 , u1+v1 , …, un+vn)
and if k is any scalar, the scalar multiple ku is
defined by
ku = (ku1 ,ku2 ,…,kun)
55
Remarks


The operations of addition and scalar
multiplication in this definition are called the
standard operations on Rn.
The zero vector in Rn is denoted by 0 and is
defined to be the vector 0 = (0, 0, …, 0).
56
Remarks


If u = (u1 ,u2 ,…,un) is any vector in Rn, then the
negative (or additive inverse) of u is denoted by
-u and is defined by -u = (-u1 ,-u2 ,…,-un).
The difference of vectors in Rn is defined by
v – u = v + (-u) = (v1 – u1 ,v2 – u2 ,…,vn – un)
57
Theorem (Properties of Vector in Rn)

If u = (u1 ,u2 ,…,un), v = (v1 ,v2 ,…, vn), and
w = (w1 ,w2 ,…, wn) are vectors in Rn and k
and l are scalars, then:




u+v=v+u
u + (v + w) = (u + v) + w
u+0=0+u=u
u + (-u) = 0; that is u – u = 0
58
Theorem (Properties of Vector in Rn)

k(lu) = (kl)u


k(u + v) = ku + kv
(k+l)u = ku+lu

1u = u
59
Euclidean Inner Product

Definition

If u = (u1 ,u2 ,…,un), v = (v1 ,v2 ,…, vn) are
vectors in Rn, then the Euclidean inner product
u · v is defined by
u · v = u1 v1 + u2 v2 + … + un vn
60
Euclidean Inner Product

Example

The Euclidean inner product of the vectors
u = (-1,3,5,7) and v = (5,-4,7,0) in R4 is
u · v = (-1)(5) + (3)(-4) + (5)(7) + (7)(0) = 18
61
Properties of Euclidean Inner
Product

Theorem 22

If u, v and w are vectors in Rn and k is any scalar,
then




u·v=v·u
(u + v) · w = u · w + v · w
(k u) · v = k(u · v)
v · v ≥ 0; Further, v · v = 0 if and only if v = 0
62
Properties of Euclidean Inner
Product

Example

(3u + 2v) · (4u + v)
= (3u) · (4u + v) + (2v) · (4u + v )
= (3u) · (4u) + (3u) · v + (2v) · (4u) + (2v) · v
=12(u · u) + 11(u · v) + 2(v · v)
63
Norm and Distance in Euclidean nSpace

We define the Euclidean norm (or Euclidean
length) of a vector u = (u1 ,u2 ,…,un) in Rn by
u  (u  u)1/ 2  u12  u22  ...  un2

Similarly, the Euclidean distance between the
points u = (u1 ,u2 ,…,un) and v = (v1 , v2 ,…,vn)
in Rn is defined by
d (u, v)  u  v  (u1  v1 ) 2  (u2  v2 ) 2  ...  (un  vn ) 2
64
Norm and Distance in Euclidean nSpace

Example

If u = (1,3,-2,7) and v = (0,7,2,2), then in the
Euclidean space R4
u  (1) 2  (3) 2  (2) 2  (7) 2  63  3 7
d (u, v )  (1  0) 2  (3  7) 2  (2  2) 2  (7  2) 2  58
65
Theorems

Theorem 23 (Cauchy-Schwarz Inequality in Rn)


If u = (u1 ,u2 ,…,un) and v = (v1 , v2 ,…,vn) are vectors
in Rn, then
|u · v| ≤ || u || || v ||
Theorem 24 (Properties of Length in Rn)

If u and v are vectors in Rn and k is any scalar, then




|| u || ≥ 0
|| u || = 0 if and only if u = 0
|| ku || = | k ||| u ||
|| u + v || ≤ || u || + || v ||
(Triangle inequality)
66
Theorems

Theorem 25 (Properties of Distance in Rn)

If u, v, and w are vectors in Rn and k is any scalar,
then




d(u, v) ≥ 0
d(u, v) = 0 if and only if u = v
d(u, v) = d(v, u)
d(u, v) ≤ d(u, w ) + d(w, v) (Triangle inequality)
67
Theorems

Theorem 26

If u, v, and w are vectors in Rn with the Euclidean
inner product, then
u · v = ¼ || u + v ||2–¼ || u–v ||2
68
Orthogonality(正交性)

Definition


Example


Two vectors u and v in Rn are called orthogonal if u · v = 0
In the Euclidean space R4 , the vectors
u = (-2, 3, 1, 4) and v = (1, 2, 0, -1) are orthogonal, since
u · v = (-2)(1) + (3)(2) + (1)(0) + (4)(-1) = 0
Theorem 27 (Pythagorean Theorem in Rn)

If u and v are orthogonal vectors in Rn which the Euclidean
inner product, then
|| u + v ||2 = || u ||2 + || v ||2
69
Matrix Formulae for the Dot
Product

If we use column matrix notation for the vectors
u = [u1 u2 … un]T and v = [v1 v2 … vn]T ,
or
u 
v 
1
u     and
un 
1
v    
vn 
then
u · v = v Tu
Au · v = u · ATv
u · Av = ATu · v
70
A Dot Product View of Matrix
Multiplication

If A = [aij] is an mr matrix and B =[bij] is an
rn matrix, then the ijth entry of AB is
ai1b1j + ai2b2j + ai3b3j + … + airbrj
which is the dot product of the ith row vector of
A and the jth column vector of B
71
A Dot Product View of Matrix
Multiplication

Thus, if the row vectors of A are r1, r2, …, rm
and the column vectors of B are c1, c2, …, cn ,
then the matrix product AB can be expressed as
r1  c1 r1  c 2  r1  c n
r  c r  c  r  c
2
n
AB   2 1 2 2
 



rm  c1 rm  c 2  rm  c n






72
Functions from Rn to R


A function is a rule f that associates with each
element in a set A one and only one element in
a set B.
If f associates the element b with the element a,
then we write b = f(a) and say that b is the
image of a under f or that f(a) is the value of f
at a.
73
Functions from Rn to R


The set A is called the domain of f and the set B
is called the codomain of f.
The subset of B consisting of all possible
values for f as a varies over A is called the
range of f.
74
Examples
Formula
f (x )
Example
f ( x)  x
Classification
2
f ( x, y )
f ( x, y)  x  y
f ( x, y , z )
f ( x, y, z )  x 2
2
2
Real-valued function
of a real variable
Function
from R to R
Real-valued function
of two real variable
Function
from R2 to
R
Real-valued function
of three real variable
Function
from R3 to
R
Real-valued function
of n real variable
Function
from Rn to
R
 y2  z2
f ( x1 , x2 ,..., xn )
f ( x1 , x2 ,..., xn ) 
x12  x22  ...  xn2
Description
75
Function from Rn to Rm


If the domain of a function f is Rn and the
codomain is Rm, then f is called a map or
transformation from Rn to Rm. We say that the
function f maps Rn into Rm, and denoted by f :
Rn  Rm.
If m = n the transformation f : Rn  Rm(=n) is
called an operator on Rn.
76
Function from Rn to Rm

Suppose f1, f2, …, fm are real-valued functions of
n real variables, say
w1 = f1(x1,x2,…,xn)
…
wm = fm(x1,x2,…,xn)
These m equations assign a unique point
(w1,w2,…,wm) in Rm to each point (x1,x2,…,xn) in
Rn and thus define a transformation from Rn to
77
Rm.
Function from Rn to Rm

If we denote this transformation by T: Rn  Rm
then
T (x1,x2,…,xn) = (w1,w2,…,wm)
78
Linear Transformations from Rn
to Rm

A linear transformation (or a linear operator if m = n)
T: Rn  Rm is defined by equations of the form
w1  a11x1  a12 x2  ...  a1n xn
w2  a21x1  a22 x2  ...  a2 n xn




wm  am1 x1  am 2 x2  ...  amn xn
or

or
 w1  a11 a12 
 w  a a 
 2    21 22
   

  
 wm  amn amn 
a1n   x1 
a2 n   x2 
 
   
 
amn   xn 
w = Ax
The matrix A = [aij] is called the standard matrix for
the linear transformation T, and T is called
multiplication by A.
79
Example (Transformation and Linear
Transformation)

The equations
w1 = x1 + x2
w2 = 3x1x2
w3 = x12 – x22
define a transformation T: R2  R3.
T(x1, x2) = (x1 + x2, 3x1x2, x12 – x22)
Thus, for example, T(1,-2) = (-1,-6,-3).
80
Remarks

Notations:


If it is important to emphasize that A is the standard matrix
for T. We denote the linear transformation T: Rn  Rm by
TA: Rn  Rm . Thus,
TA(x) = Ax
We can also denote the standard matrix for T by the
symbol [T], or
T(x) = [T]x
81
Remarks

Remark:

We have establish a correspondence between mn
matrices and linear transformations from Rn to Rm :

To each matrix A there corresponds a linear transformation TA
(multiplication by A), and to each linear transformation T: Rn  Rm,
there corresponds an mn matrix [T] (the standard matrix for T).
82
Examples

Zero Transformation from Rn to Rm


If 0 is the mn zero matrix and 0 is the zero vector in
Rn, then for every vector x in Rn
T0(x) = 0x = 0
So multiplication by zero maps every vector in Rn
into the zero vector in Rm. We call T0 the zero
transformation from Rn to Rm.
83
Examples

Identity Operator on Rn



If I is the nn identity, then for every vector in Rn
TI(x) = Ix = x
So multiplication by I maps every vector in Rn into
itself.
We call TI the identity operator on Rn.
84
Projection Operators


In general, a projection operator (or more
precisely an orthogonal projection operator)
on R2 or R3 is any operator that maps each
vector into its orthogonal projection on a
line or plane through the origin.
The projection operators are linear.
85
Projection Operators
86
Projection Operators
87
Compositions of Linear
Transformations


If TA : Rn  Rk and TB : Rk  Rm are linear
transformations, then for each x in Rn one can first
compute TA(x), which is a vector in Rk, and then one
can compute TB(TA(x)), which is a vector in Rm.
Thus, the application of TA followed by TB produces
a transformation from Rn to Rm.
88
Compositions of Linear
Transformations




This transformation is called the composition of TB
with TA and is denoted by TB ◦ TA. Thus
(TB ◦ TA)(x) = TB(TA(x))
The composition TB ◦ TA is linear since
(TB ◦ TA)(x) = TB(TA(x)) = B(Ax) = (BA)x
The standard matrix for TB ◦ TA is BA. That is,
TB ◦ TA = TBA
Multiplying matrices is equivalent to composing the
corresponding linear transformations in the right-toleft order of the factors.
89
Compositions of Three or More
Linear Transformations


Consider the linear transformations
T1 : Rn  Rk , T2 : Rk  Rl , T3 : Rl  Rm
We can define the composition (T3◦T2◦T1) : Rn  Rm
by
(T3◦T2◦T1)(x) : T3(T2(T1(x)))
90
Compositions of Three or More
Linear Transformations


This composition is a linear transformation and the
standard matrix for T3◦T2◦T1 is related to the standard
matrices for T1,T2, and T3 by
[T3◦T2◦T1] = [T3][T2][T1]
If the standard matrices for T1, T2, and T3 are denoted
by A, B, and C, respectively, then we also have
TC◦TB◦TA = TCBA
91
One-to-One Linear
transformations

Definition


A linear transformation T : Rn →Rm is said to be
one-to-one if T maps distinct vectors (points) in
Rn into distinct vectors (points) in Rm
Remark:

That is, for each vector w in the range of a oneto-one linear transformation T, there is exactly
one vector x such that T(x) = w.
92
Theorem (Equivalent Statements)

If A is an nn matrix and TA : Rn  Rn is
multiplication by A, then the following
statements are equivalent.



A is invertible
The range of TA is Rn
TA is one-to-one
93
Examples

The rotation operator T : R2  R2 is one-toone

The standard matrix for T is
cos 
[T ]  
sin 

 sin  
cos  
[T] is not invertible since
cos 
det
sin 
 sin 
 cos 2   sin 2   1  0
cos 
94
Examples

The projection operator T : R3  R3 is not
one-to-one

The standard matrix for T is
1 0 0 
[T ]  0 1 0 
0 0 0

[T] is invertible since det[T] = 0
95
Inverse of a One-to-One Linear
Operator

Suppose TA : Rn  Rn is a one-to-one linear
operator
 The matrix A is invertible.
 TA-1 : Rn  Rn is itself a linear operator; it is
called the inverse of TA.
 TA(TA-1(x)) = AA-1x = Ix = x and
TA-1(TA (x)) = A-1Ax = Ix = x
 TA ◦ TA-1 = TAA-1 = TI and TA-1 ◦ TA = TA-1A = TI
96
Inverse of a One-to-One Linear
Operator



If w is the image of x under TA, then TA-1
maps w back into x, since
TA-1(w) = TA-1(TA (x)) = x
When a one-to-one linear operator on Rn is
written as T : Rn  Rn, then the inverse of
the operator T is denoted by T-1.
Thus, by the standard matrix, we have
[T-1]=[T]-1
97
Example

Let T : R2  R2 be the operator that rotates each
vector in R2 through the angle :
cos 
[T ]  
sin 

 sin  
cos  
Undo the effect of T means rotate each vector in R2
through the angle -.
98
Example

This is exactly what the operator T-1 does: the
standard matrix T-1 is
 cos 
[T ]  [T ]  
 sin 
1

1
sin   cos(  )


cos   sin(  )
 sin(  ) 
cos(  )
The only difference is that the angle  is replaced by 
99
Example

Show that the linear operator T : R2  R2 defined by
the equations
w1= 2x1+ x2
w2 = 3x1+ 4x2
is one-to-one, and find T-1(w1,w2).
100
Example

Solution:
w1  2 1   x1 
w   3 4  x 
 2 
 2 
 4
 5
w
1  1 
[T ]   
 w2   3
 5
T 1 ( w1 , w2 )  (
2 1 
[T ]  

3 4
 4
 5
1
1
[T ]  [T ]  
 3
 5
1
 
5

2
5 
1
1 
 4
 
w

w2 
1

w


5
5
5
1
   

2   w2   3
2 
 w1  w2


5
5 
 5
4
1
3
2
w1  w2 , w1  w2 )
5
5
5
5
101
Linearity Properties

Theorem 28 (Properties of Linear
Transformations)

A transformation T : Rn  Rm is linear if and only if
the following relationships hold for all vectors u and
v in Rn and every scalar c.


T(u + v) = T(u) + T(v)
T(cu) = cT(u)
102
Linearity Properties

Theorem 29

If T : Rn  Rm is a linear transformation, and e1,
e2, …, en are the standard basis vectors for Rn, then
the standard matrix for T is
A = [T] = [T(e1) | T(e2) | … | T(en)]
103
Example (Standard Matrix for a
Projection Operator)

Let l be the line in the xy-plane that passes through the
origin and makes an angle  with the positive x-axis,
where 0 ≤  ≤ . Let T: R2  R2 be a linear operator that
maps each vector into orthogonal projection on l.


Find the standard matrix for T.
Find the orthogonal projection of
the vector x = (1,5) onto the line
through the origin that makes an
angle of  = /6 with the positive
x-axis.
104
Example


The standard matrix for T can be written as
[T] = [T(e1) | T(e2)]
Consider the case 0    /2.

||T(e1)|| = cos   norm of T(e1)
 T (e1 ) cos    cos 2  
T (e1 )  


sin

cos

T
(
e
)
sin


 

1

||T(e2)|| = sin 
 T (e 2 )  cos   sin  cos  
T (e 2 )  


2
sin

T
(
e
)

sin


 

2
2
 cos  sin  cos  
T   

sin 2  
sin  cos 
105
 cos 2  sin  cos  
T   

sin 2  
sin  cos 
Example

Since sin (/6) = 1/2 and cos (/6) = 3 /2, it
follows from part (a) that the standard matrix
for this projection operator is
3 4
3 4
[T ]  

 3 4 1 4 
Thus,
3  5 3 
 1   3 4
3 4 1   4 

T      
   
 5   3 4 1 4  5  3  5 


 4

106
Theorem (Equivalent Statements)

If A is an nn matrix, and if TA : Rn  Rn is
multiplication by A, then the following are
equivalent.




A is invertible
Ax = 0 has only the trivial solution
The reduced row-echelon form of A is In
A is expressible as a product of elementary
matrices
107
Theorem (Equivalent Statements)





Ax = b is consistent for every n1 matrix b
Ax = b has exactly one solution for every n1
matrix b
det(A)  0
The range of TA is Rn
TA is one-to-one
108
Example (Multiple Linear
Regression)(1/3)

Given n vectors u1, u2, …,un, sampling from a
population to fit the multiple regression,
y  0  1 x1   2 x2     m xm  

that is,
u i  Xi
Yi  i  1,2,..., n where
Xi  1 xi1
Yi   yi 
xi 2  xim 
109
Example (Multiple Linear
Regression)(2/3)

We then can name the following matrices:
 X1 
 X  1
 2  1
X  X3   
  
   1
 Xn  
and the ith
x11  x1m 
 y1 
y 
x21  x2 m 
, Y   2

   

 
xn1  xnm 
 yn 
residual
m
^
ri  yi   xij  j
j 1
110
Example (Multiple Linear
Regression)(3/3)

The best fit is obtained when the sum of
squared residuals is minimized. From the theory
of linear least squares, the parameter estimators
are found by solving the normal equations:
 x x   x y
That is,
n
m
i 1 k 1

n
^
ij ik
k
11
ij
i
X Xβ  X Y
β  X X  X Y
T
^
^
T
T
1
T
111
General Vector Spaces
Definition (Vector Space)

Let V be an arbitrary nonempty set of
objects on which two operations are defined:



Addition
Multiplication by scalars
If the following axioms are satisfied by all
objects u, v, w in V and all scalars k and l,
then we call V a vector space and we call
the objects in V vectors.
113
Definition (Vector Space)
1. If u and v are objects in V, then u + v is in V.
2. u + v = v + u
3. u + (v + w) = (u + v) + w
4. There is an object 0 in V, called a zero vector for V,
such that 0 + u = u + 0 = u for all u in V.
5. For each u in V, there is an object -u in V, called a
negative of u, such that u + (-u) = (-u) + u = 0.
6. If k is any scalar and u is any object in V, then ku is
in V.
114
Definition (Vector Space)
7. k (u + v) = ku + kv
8. (k + l) u = ku + lu
9. k (lu) = (kl) (u)
10. 1u = u
115
Remarks

Depending on the application, scalars may be
real numbers or complex numbers.

Vector spaces in which the scalars are complex
numbers are called complex vector spaces, and those
in which the scalars must be real are called real
vector spaces.
116
Remarks

The definition of a vector space specifies neither
the nature of the vectors nor the operations.


Any kind of object can be a vector, and the
operations of addition and scalar multiplication may
not have any relationship or similarity to the
standard vector operations on Rn.
The only requirement is that the ten vector space
axioms be satisfied.
117
Example (Rn Is a Vector Space)



The set V = Rn with the standard operations of
addition and scalar multiplication is a vector
space.
Axioms 1 and 6 follow from the definitions of
the standard operations on Rn; the remaining
axioms follow from other Theorems
The three most important special cases of Rn are
R (the real numbers), R2 (the vectors in the
plane), and R3 (the vectors in 3-space).
118
Example (22 Matrices)

Show that the set V of all 22 matrices with real
entries is a vector space if vector addition is
defined to be matrix addition and vector scalar
multiplication is defined to be matrix scalar
multiplication.
119
Example (22 Matrices)

Let

To prove Axiom 1, we must show that u + v is an
object in V; that is, we must show that u + v is a
22 matrix.
u11 u12 
u

u
u
 21 22 
and
v11 v12 
v

v
v
 21 22 
u11 u12  v11 v12   u11  v11 u12  v12 
uv  





u
u
v
v
u

v
u

v
 21 22   21 22   21 21 22 22 
120
Example

Similarly, Axiom 6 hold because for any real
number k we have
u11 u12   ku11 ku12 
ku  k 



u
u
ku
ku
 21 22   21
22 

so that ku is a 22 matrix and consequently is an
object in V.
Axioms 2 follows from Theorem 1.4.1a since
u11 u12  v11 v12  v11 v12  u11 u12 
uv  



 vu




u21 u22  v21 v22  v21 v22  u21 u22 
121
Example

Similarly, Axiom 3 follows from part (b) of that
theorem; and Axioms 7, 8, and 9 follow from part
(h), (j), and (l), respectively.
122
Example

0 0 
To prove Axiom 4, let 0  

0
0


Then
0 0 u11 u12  u11 u12 
0u  


u



0 0 u21 u22  u21 u22 
Similarly, u + 0 = u.
123
Example

  u11  u12 
To prove Axiom 5, let  u  


u

u
22 
 21
Then

u11 u12    u11  u12  0 0
u  (u)  


0



u21 u22   u21  u22  0 0
Similarly, (-u) + u = 0.
For Axiom 10, 1u = u.
124
Example (Vector Space of mn
Matrices)


The previous example is a special case of a
more general class of vector spaces.
The arguments in that example can be
adapted to show that the set V of all mn
matrices with real entries, together with the
operations matrix addition and scalar
multiplication, is a vector space.
125
Example (Vector Space of mn
Matrices)


The mn zero matrix is the zero vector 0,
and if u is the mn matrix U, then matrix –
U is the negative –u of the vector u.
We shall denote this vector space by the
symbol Mmn
126
Example (Not a Vector Space)

Let V = R2 and define addition and scalar
multiplication operations as follows: If u = (u1,
u2) and v = (v1, v2), then define
u + v = (u1 + v1, u2 + v2)
and if k is any real number, then define
k u = (k u1, 0)
130
Example (Not a Vector Space)


There are values of u for which Axiom 10 fails
to hold. For example, if u = (u1, u2) is such that
u2 ≠ 0,then
1u = 1 (u1, u2) = (1 u1, 0) = (u1, 0) ≠ u
Thus, V is not a vector space with the stated
operations.
131
Every Plane Through the Origin Is a
Vector Space

Check all the axioms!


Let V be any plane through the origin in R3. Since R3
itself is a vector space, Axioms 2, 3, 7, 8, 9, and 10 hold
for all points in R3 and consequently for all points in the
plane V.
We need only show that Axioms 1, 4, 5, and 6 are
satisfied.
132
Every Plane Through the Origin Is a
Vector Space

Check all the axioms!



Since the plane V passes through the origin, it has an
equation of the form ax + by + cz = 0. If u = (u1, u2, u3)
and v = (v1, v2, v3) are points in V, then au1 + bu2 + cu3 =
0 and av1 + bv2 + cv3 = 0. Adding these equations gives
a(u1 + v1) +b(u2 + v2) +c (u3 + v3) = 0.
Axiom 1: u + v = (u1 + v1, u2 + v2, u3 + v3); thus u + v
lies in the plane V.
Axioms 5: Multiplying au1 + bu2 + cu3 = 0 through by -1
gives a(-u1) + b(-u2) + c(-u3) = 0 ; thus, -u = (-u1, -u2, -u3)
lies in V.
133
The Zero Vector Space


Let V consist of a signle object, which we
denote by 0, and define 0 + 0 = 0 and k 0 =
0 for all scalars k.
We called this the zero vector space.
134
Theorem

Let V be a vector space, u be a vector in V,
and k a scalar; then:
 0 u = 0
 k 0 = 0
 (-1) u = -u
 If k u = 0 , then k = 0 or u = 0.
135
Subspaces

Definition


A subset W of a vector space V is called a subspace
of V if W is itself a vector space under the addition
and scalar multiplication defined on V.
Theorem 32

If W is a set of one or more vectors from a vector
space V, then W is a subspace of V if and only if the
following conditions hold:
a)If u and v are vectors in W, then u + v is in W.
136
b)If k is any scalar and u is any vector in W , then ku is in W.
Subspaces

Remark

Theorem 32 states that W is a subspace of V if and
only if W is a closed under addition (condition (a))
and closed under scalar multiplication (condition (b)).
137
Example

Let W be any plane through
the origin and let u and v be
any vectors in W.

u + v must lie in W since it is
the diagonal of the
parallelogram determined by
u and v, and k u must line in
W for any scalar k since k u
lies on a line through u.
138
Example

Thus, W is closed under
addition and scalar
multiplication, so it is a
subspace of R3.
139
Example


A line through the origin of R3 is a subspace
of R3.
Let W be a line through the origin of R3.
140
Example (Not a Subspace)

Let W be the set of all
points (x, y) in R2
such that x  0 and y
 0. These are the
points in the first
quadrant.
141
Example (Not a Subspace)


The set W is not a
subspace of R2 since
it is not closed under
scalar multiplication.
For example, v = (1, 1)
lines in W, but its
negative (-1)v = -v =
(-1, -1) does not.
142
Remarks

Think about “set” and “empty set”!
Every nonzero vector space V has at least two
subspace: V itself is a subspace, and the set {0}
consisting of just the zero vector in V is a
subspace called the zero subspace.
143
Remarks

Examples of subspaces of R2 and R3:

Subspaces of R2:




{0}
Lines through the origin
R2
Subspaces of R3:





Think about “set” and “empty set”!
{0}
Lines through the origin
Planes through origin
R3
They are actually the only subspaces of R2 and
R3
144
Solution Space

Solution Space of Homogeneous Systems


If Ax = b is a system of the linear equations, then
each vector x that satisfies this equation is called a
solution vector of the system.
Theorem 33 shows that the solution vectors of a
homogeneous linear system form a vector space,
which we shall call the solution space of the system.
148
Solution Space

Theorem 33
 If Ax = 0 is a homogeneous linear system of m
equations in n unknowns, then the set of
solution vectors is a subspace of Rn.
149
Example

Find the solution spaces of the linear systems.
1 - 2 3   x 
0
  y   0
(a) 
2
4
6

  
 


3 - 6 9 
 
z 

0

1 - 2 3   x 
0
  y   0
(c) 
-3
7
-8

  
 


4 1 2 
 
z 

0



1 - 2 3   x 
 0
  y    0
(b) 
-3
7
8

  
 


-2 4 -6 
 
z 

 0

0 0 0   x 
0
  y   0
(d) 
0
0
0

  
 


0 0 0 
 
z 

0

Each of these systems has three unknowns, so the
solutions form subspaces of R3.
Geometrically, each solution space must be a line
through the origin, a plane through the origin, the
origin only, or all of R3.
150
Example
Solution.
(a) x = 2s - 3t, y = s, z = t
x = 2y - 3z or x – 2y + 3z = 0
This is the equation of the plane through the origin with
n = (1, -2, 3) as a normal vector.
(b) x = -5t , y = -t, z =t
which are parametric equations for the line through the origin
parallel to the vector v = (-5, -1, 1).
(c) The solution is x = 0, y = 0, z = 0, so the solution space is the
origin only, that is {0}.
(d) The solution are x = r , y = s, z = t, where r, s, and t have
151
arbitrary values, so the solution space is all of R3.
Linear Combination

Definition

A vector w is a linear combination of the vectors v1,
v2,…, vr if it can be expressed in the form w = k1v1
+ k2v2 + · · · + kr vr where k1, k2, …, kr are scalars.
152
Linear Combination

Vectors in R3 are linear combinations of i, j, and k
3
 Every vector v = (a, b, c) in R is expressible as a
linear combination of the standard basis vectors
i = (1, 0, 0), j = (0, 1, 0), k = (0, 0, 1)
since
v = a(1, 0, 0) + b(0, 1, 0) + c(0, 0, 1) = a i + b j + c k
153
Linear Combination and
Spanning

Theorem 34

If v1, v2, …, vr are vectors in a vector space V, then:
 The set W of all linear combinations of v1,
v2, …, vr is a subspace of V.
 W is the smallest subspace of V that contain v1,
v2, …, vr in the sense that every other subspace
of V that contain v1, v2, …, vr must contain W.
158
Linear Combination and
Spanning

Definition


If S = {v1, v2, …, vr} is a set of vectors in a vector
space V, then the subspace W of V containing of all
linear combination of these vectors in S is called
the space spanned by v1, v2, …, vr, and we say that
the vectors v1, v2, …, vr span W.
To indicate that W is the space spanned by the
vectors in the set S = {v1, v2, …, vr}, we write W =
span(S) or W = span{v1, v2, …, vr}.
159
Example

If v1 and v2 are non-collinear vectors in R3 with their
initial points at the origin, then span{v1, v2}, which
consists of all linear combinations k1v1 + k2v2 is the
plane determined by v1 and v2.
160
Example

Similarly, if v is a nonzero vector in R2 and R3, then
span{v}, which is the set of all scalar multiples kv, is
the linear determined by v.
161
Example

Determine whether v1 = (1, 1, 2), v2 = (1, 0, 1),
and v3 = (2, 1, 3) span the vector space R3.
162
Example

Solution


Is it possible that an arbitrary vector b = (b1, b2, b3) in R3
can be expressed as a linear combination b = k1v1 + k2v2
+ k3v3 ?
b = (b1, b2, b3) = k1(1, 1, 3) + k2(1, 0, 1) + k3(2, 1, 3) =
(k1+k2+2k3, k1+k3, 2k1+k2+3k3) or
k1 + k2 + 2k3 = b1
k1
+ k3 = b2
2k1 + k2 + 3 k3 = b3
163
Example

Solution

This system is consistent for all values of b1, b2, and b3 if
and only if the coefficient matrix
1 1 2 
A  1 0 1 
 2 1 3 
has a nonzero determinant.

However, det(A) = 0, so that v1, v2, and v3, do not span R3.
164
Theorem

If S = {v1, v2, …, vr} and S = {w1, w2, …,
wr} are two sets of vector in a vector space
V, then
span{v1, v2, …, vr} = span{w1, w2, …, wr}
if and only if each vector in S is a linear
combination of these in S and each vector
in S is a linear combination of these in S.
165
Linearly Dependent &
Independent

Definition


If S = {v1, v2, …, vr} is a nonempty set of
vector, then the vector equation k1v1 + k2v2 + …
+ krvr = 0 has at least one solution, namely k1 =
0, k2 = 0, … , kr = 0.
If this the only solution, then S is called a
linearly independent set. If there are other
solutions, then S is called a linearly dependent
set.
166
Linearly Dependent &
Independent

Examples


If v1 = (2, -1, 0, 3), v2 = (1, 2, 5, -1), and v3 = (7,
-1, 5, 8).
Then the set of vectors S = {v1, v2, v3} is
linearly dependent, since 3v1 + v2 – v3 = 0.
167
Example

Let i = (1, 0, 0), j = (0, 1, 0), and k = (0, 0, 1) in R3.


Consider the equation k1i + k2j + k3k = 0
 k1(1, 0, 0) + k2(0, 1, 0) + k3(0, 0, 1) = (0, 0, 0)
 (k1, k2, k3) = (0, 0, 0)
 The set S = {i, j, k} is linearly independent.
Similarly the vectors
e1 = (1, 0, 0, …,0), e2 = (0, 1, 0, …, 0),
…, en = (0, 0, 0, …, 1)
form a linearly independent set in Rn.
168
Example

Remark:

To check whether a set of vectors is linear
independent or not, write down the linear
combination of the vectors and see if their
coefficients all equal zero.
169
Example

Determine whether the vectors
v1 = (1, -2, 3), v2 = (5, 6, -1), v3 = (3, 2, 1)
form a linearly dependent set or a linearly
independent set.
170
Example

Solution

Let the vector equation k1v1 + k2v2 + k3v3 = 0
 k1(1, -2, 3) + k2(5, 6, -1) + k3(3, 2, 1) = (0, 0, 0)

k1 + 5k2 + 3k3 = 0
-2k1 + 6k2 + 2k3 = 0
3k1 – k2 + k3 = 0
 det(A) = 0
 The system has nontrivial solutions
 v1,v2, and v3 form a linearly dependent set
171
Theorems

Theorem 36

A set with two or more vectors is:
Linearly dependent if and only if at least one of the
vectors in S is expressible as a linear combination of
the other vectors in S.
 Linearly independent if and only if no vector in S is
expressible as a linear combination of the other
vectors in S.

172
Theorems
Theorem 37



A finite set of vectors that contains the zero
vector is linearly dependent.
A set with exactly two vectors is linearly
independently if and only if neither vector is a
scalar multiple of the other.
173
Theorems
Theorem 38


Let S = {v1, v2, …, vr} be a set of vectors in Rn.
If r > n, then S is linearly dependent.
174
Geometric Interpretation of Linear
Independence


In R2 and R3, a set of two vectors is linearly
independent if and only if the vectors do not lie on the
same line when they are placed with their initial points
at the origin.
In R3, a set of three vectors is linearly independent if
and only if the vectors do not lie in the same plane
when they are placed with their initial points at the
origin.
178
Basis

Definition

If V is any vector space and S = {v1, v2, …,vn}
is a set of vectors in V, then S is called a basis
for V if the following two conditions hold:


S is linearly independent.
S spans V.
182
Basis

Theorem 39 (Uniqueness of Basis
Representation)

If S = {v1, v2, …,vn} is a basis for a vector
space V, then every vector v in V can be
expressed in the form
v = c1v1 + c2v2 + … + cnvn
in exactly one way.
183
Coordinates Relative to a Basis


If S = {v1, v2, …, vn} is a basis for a vector space V,
and
v = c1v1 + c2v2 + ··· + cnvn
is the expression for a vector v in terms of the basis S,
then the scalars c1, c2, …, cn, are called the coordinates
of v relative to the basis S.
The vector (c1, c2, …, cn) in Rn constructed from these
coordinates is called the coordinate vector of v relative
to S; it is denoted by
(v)S = (c1, c2, …, cn)
184
Coordinates Relative to a Basis

Remark:
 Coordinate vectors depend not only on the basis S but
also on the order in which the basis vectors are
written.
 A change in the order of the basis vectors results in a
corresponding change of order for the entries in the
coordinate vector.
185
Example (Standard Basis for R3)

Suppose that i = (1, 0, 0), j = (0, 1, 0), and k = (0,
0, 1), then S = {i, j, k} is a linearly independent
set in R3.
This set also spans R3 since any vector v = (a, b, c)
in R3 can be written as
v = (a, b, c) = a(1, 0, 0) + b(0, 1, 0) + c(0, 0, 1) = ai
+ bj + ck

186
Example (Standard Basis for R3)



Thus, S is a basis for R3; it is called the standard
basis for R3.
Looking at the coefficients of i, j, and k, it
follows that the coordinates of v relative to
the standard basis are a, b, and c, so
(v)S = (a, b, c)
Comparing this result to v = (a, b, c), we have
v = (v)S
187
Standard Basis for Rn



If e1 = (1, 0, 0, …, 0), e2 = (0, 1, 0, …, 0), …, en
= (0, 0, 0, …, 1), then
S = {e1, e2, …, en}
is a linearly independent set in Rn.
This set also spans Rn since any vector v = (v1,
v2, …, vn) in Rn can be written as
v = v1e1 + v2e2 + … + vnen
Thus, S is a basis for Rn; it is called the standard
basis for Rn.
188
Standard Basis for Rn


The coordinates of v = (v1, v2, …, vn) relative to
the standard basis are v1, v2, …, vn, thus
(v)S = (v1, v2, …, vn)
As the previous example, we have v = (v)s, so a
vector v and its coordinate vector relative to the
standard basis for Rn are the same.
189
Example

Let v1 = (1, 2, 1), v2 = (2, 9, 0), and v3 = (3, 3, 4).
Show that the set S = {v1, v2, v3} is a basis for R3.
190
Example

Solution:


To show that the set S spans R3, we must show that an arbitrary
vector
b = (b1, b2, b3)
can be expressed as a linear combination
b = c1v1 + c2v2 + c3v3
of the vectors in S.
Let (b1, b2, b3) = c1(1, 2, 1) + c2(2, 9, 0) + c3(3, 3, 4)

c1 +2c2 +3c3 = b1
2c1+9c2 +3c3 = b2
c1
+4c3 = b3
 det(A)  0
 S is a basis for R3
191
Example (Representing a Vector
Using Two Bases)

Let S = {v1, v2, v3} be the basis for R3 in the
preceding example.


Find the coordinate vector of v = (5, -1, 9) with
respect to S.
Find the vector v in R3 whose coordinate vector with
respect to the basis S is (v)s = (-1, 3, 2).
192
Example (Representing a Vector
Using Two Bases)

Solution (a)




We must find scalars c1, c2, c3 such that v = c1v1 +
c2v2 + c3v3, or, in terms of components, (5, -1, 9) =
c1(1, 2, 1) + c2(2, 9, 0) + c3(3, 3, 4)
Solving this, we obtaining c1 = 1, c2 = -1, c3 = 2.
Therefore, (v)s = (1, -1, 2).
Solution (b)

Using the definition of the coordinate vector (v)s, we
obtain
v = (-1)v1 + 3v2 + 2v3 = (11, 31, 7).
193
Standard Basis for Pn

S = {1, x, x2, …, xn} is a basis for the
vector space Pn of polynomials of the form
a0 + a1x + … + anxn. The set S is called the
standard basis for Pn.
Find the coordinate vector of the
polynomial p = a0 + a1x + a2x2 relative to
the basis S = {1, x, x2} for P2 .
194
Standard Basis for Pn

Solution:

The coordinates of p = a0 + a1x + a2x2 are the
scalar coefficients of the basis vectors 1, x,
and x2, so
(p)s=(a0, a1, a2).
195
Standard Basis for Mmn



Let M1  1
0
0 1 
0 0 
0 0 
,
M

,
M

,
M

2
3
4

0 0 
1 0 
0 1 
0 0 






The set S = {M1, M2, M3, M4} is a basis for the vector
space M22 of 2×2 matrices.
To see that S spans M22, note that an arbitrary vector
(matrix)  a b  can be written as
c

d 
a b 
1 0 0 1 0 0
0 0

a

b

c

d
c d 
0 0 0 0 1 0
0 1  aM 1  bM 2  cM 3  dM 4



 
 



196
Standard Basis for Mmn

To see that S is linearly independent, assume aM1 +
bM2 + cM3 + dM4 = 0. It follows that a b  0 0
 c d   0 0 

 

Thus, a = b = c = d = 0, so S is lin. indep.


The basis S is called the standard basis for M22.
More generally, the standard basis for Mmn consists
of the mn different matrices with a single 1 and
zeros for the remaining entries.
197
Basis for the Subspace span(S)

If S = {v1, v2, …,vn} is a linearly
independent set in a vector space V, then S
is a basis for the subspace span(S)
since the set S span span(S) by definition of
span(S).
198
Finite-Dimensional

Definition

A nonzero vector V is called finite-dimensional
if it contains a finite set of vector {v1, v2, …,vn}
that forms a basis. If no such set exists, V is
called infinite-dimensional. In addition, we
shall regard the zero vector space to be finitedimensional.
199
Finite-Dimensional

Example


The vector spaces Rn, Pn, and Mmn are finitedimensional.
The vector spaces F(-, ), C(- , ), Cm(- ,
), and C∞(- , ) are infinite-dimensional.
200
Theorems

Theorem 40

Let V be a finite-dimensional vector space and
{v1, v2, …,vn} any basis.


If a set has more than n vector, then it is linearly
dependent.
If a set has fewer than n vector, then it does not span
V.
201
Theorems

Theorem 41

All bases for a finite-dimensional vector space
have the same number of vectors.
202
Dimension

Definition


The dimension of a finite-dimensional vector
space V, denoted by dim(V), is defined to be the
number of vectors in a basis for V.
We define the zero vector space to have
dimension zero.
203
Dimension

Dimensions of Some Vector Spaces:



dim(Rn) = n [The standard basis has n vectors]
dim(Pn) = n + 1 [The standard basis has n + 1
vectors]
dim(Mmn) = mn [The standard basis has mn
vectors]
204
Example

Determine a basis for and the dimension of the
solution space of the homogeneous system
2x1 + 2x2 – x3
+ x5 = 0
-x1 + x2 + 2x3 – 3x4 + x5 = 0
x1 + x2 – 2x3
– x5 = 0
x3+ x4 + x5 = 0
205
Example

Solution:


The general solution of the given system is
x1 = -s-t, x2 = s,
x3 = -t, x4 = 0, x5 = t
Therefore, the solution vectors can be written as
 x1   s  t   1  1
x  
    
 2  s   1   0 
 x3     t   s  0   t  1
  
    
0
x
 4 
 0 0
 x5   t   0   1 
 
206
Example

Which shows that the vectors
 1
1
 
v1   0  and
 
0
 0 

 1
0
 
v 2   1
 
0
 1 
span the solution space.
Since they are also linearly independent, {v1,
v2} is a basis , and the solution space is twodimensional.
207
Theorems

Theorem 42 (Plus/Minus Theorem)

Let S be a nonempty set of vectors in a vector space
V.


If S is a linearly independent set, and if v is a vector in V that is
outside of span(S), then the set S  {v} that results by inserting
v into S is still linearly independent.
If v is a vector in S that is expressible as a linear combination
of other vectors in S, and if S – {v} denotes the set obtained by
removing v from S, then S and S – {v} span the same space;
that is, span(S) = span(S – {v})
208
Theorems

Theorem 43

If V is an n-dimensional vector space, and if S is a set
in V with exactly n vectors, then S is a basis for V if
either S spans V or S is linearly independent.
209
Example


Show that v1 = (-3, 7) and v2 = (5, 5) form a basis for
R2 by inspection.
Solution:
 Neither vector is a scalar multiple of the other
 The two vectors form a linear independent set in
the 2-D space R2
 The two vectors form a basis by Theorem 5.4.5.
210
Example


Show that v1 = (2, 0, 1) , v2 = (4, 0, 7), v3 = (-1, 1, 4)
form a basis for R3 by inspection.
Solution:
 The vectors v1 and v2 form a linearly independent set
in the xz-plane.
 The vector v3 is outside of the xz-plane, so the set {v1,
v2 , v3} is also linearly independent.
3
 Since R is three-dimensional, Theorem 5.4.5 implies
that {v1, v2 , v3} is a basis for R3.
211
Theorems

Theorem 44

Let S be a finite set of vectors in a finitedimensional vector space V.


If S spans V but is not a basis for V, then S can be
reduced to a basis for V by removing appropriate
vectors from S.
If S is a linearly independent set that is not already a
basis for V, then S can be enlarged to a basis for V
by inserting appropriate vectors into S.
212
Theorems

Theorem 45


If W is a subspace of a finite-dimensional
vector space V, then dim(W)  dim(V).
If dim(W) = dim(V), then W = V.
213
Definition

For an mn matrix
the vectors
 a11 a12  a1n 
a

a

a
21
22
2
n

A
 

 


am1 am 2  amn 
r1  [a11 a12  a1n ]
r2  [a21 a22  a2 n ]

rm  [am1
am 2  amn ]
in Rn formed form the rows of A are called the row
vectors of A, and the vectors
 a11 
 a12 
 a1n 
a 
a 
a 
21
22
c1   , c 2   ,, c n   2 n 
  
  
  
 
 
 
am1 
 am 2 
amn 
in Rm formed from the columns of A are called the
column vectors of A.
214
Example


2 1 0
Let A  

3

1
4


The row vectors of A are
r1 = [2 1 0] and r2 = [3 -1 4]
and the column vectors of A are
 2
1
0
c1    , c2    , and c3   
 3
 1
 4
215
Row Space and Column Space

Definition


If A is an mn matrix, then the subspace of Rn
spanned by the row vectors of A is called the row
space of A, and the subspace of Rm spanned by the
column vectors is called the column space of A.
The solution space of the homogeneous system of
equation Ax = 0, which is a subspace of Rn, is called
the nullspace of A.
Amn
 a11 a12  a1n 
a

a

a
21
22
2
n


 

 


a
a

a
m2
mn 
 m1
 a11 
 a12 
 a1n 
a 
a 
a 
21
22
c1   , c 2   ,, c n   2 n 
  
  
  
 
 
 
a
a
 m1 
 m2 
amn 
216
Row Space and Column Space

Theorem 46

A system of linear equations Ax = b is consistent if
and only if b is in the column space of A.
217
Example

Let Ax = b be the linear system
 1 3 2   x1   1 
 1 2 3  x    9

 2  
 2 1 2  x3   3
Show that b is in the column space of A, and
express b as a linear combination of the column
vectors of A.
218
Example

Solution:



Solving the system by Gaussian elimination yields
x1 = 2, x2 = -1, x3 = 3
Since the system is consistent, b is in the column
space of A.
Moreover, it follows that
 1  3  2   1 
2  1    2  3  3   9
 2  1   2  3
219
General and Particular Solutions

Theorem 47

If x0 denotes any single solution of a consistent
linear system Ax = b, and if v1, v2, …, vk form a
basis for the nullspace of A, (that is, the
solution space of the homogeneous system Ax =
0), then every solution of Ax = b can be
expressed in the form
x = x0 + c1v1 + c2v2 + · · · + ckvk
Conversely, for all choices of scalars c1, c2, …,
ck the vector x in this formula is a solution of
Ax = b.
220
General and Particular Solutions

Remark



The vector x0 is called a particular solution of
Ax = b.
The expression x0 + c1v1 + · · · + ckvk is called
the general solution of Ax = b, the expression
c1v1 + · · · + ckvk is called the general solution
of Ax = 0.
The general solution of Ax = b is the sum of
any particular solution of Ax = b and the
general solution of Ax = 0.
221
Example (General Solution of Ax
= b)

The solution to the nonhomogeneous
system
x1 + 3x2 – 2x3
+ 2x5
=0
2x1 + 6x2 – 5x3 – 2x4 + 4x5 – 3x6 = -1
5x3 + 10x4
+ 15x6 = 5
2x1 + 5x2
+ 8x4 + 4x5 + 18x6 = 6
is
x1 = -3r - 4s - 2t, x2 = r,
x3 = -2s, x4 = s,
x5 = t, x6 = 1/3
222
Example (General Solution of Ax
= b)

The result can be written in vector form as
 x1   3r  4 s  2t   0   3  4  2
x  
  0  1 0 0
r
2
  
        
 x3  
  0   0    2  0 
 2s
 
     r    s   t  
x
s
4
  
  0  0 1 0
 x5  
  0  0 0 1
t
  
        
1/ 3
 1 / 3  0   0   0 
 x6  
 


x0
x
which is the general solution.

The vector x0 is a particular solution of
nonhomogeneous system, and the linear
combination x is the general solution of the
homogeneous system.
223
Example

Find a basis for the nullspace of
 2 2 1 0 1 
 1 1 2 3 1 

A
 1 1 2 0 1


0
0
1
1
1


224
Example

Solution


The nullspace of A is the solution space of the homogeneous system
2x1 + 2x2 – x3
+ x5 = 0
-x1 – x2 – 2 x3 – 3x4 + x5 = 0
x1 + x2 – 2 x3
– x5 = 0
x3 + x 4 + x 5 = 0
In Example 10 of Section 5.4 we showed that the vectors
 1
 1
1
0
 
 
v1   0  and v 2   1
 
 
0
 
0
 0 
 1 
form a basis for the nullspace.
225
Theorems

Theorem 48


Elementary row operations do not change both
the nullspace and row space of a matrix.
Theorem 49

If A and B are row equivalent matrices, then:


A given set of column vectors of A is linearly
independent if and only if the corresponding column
vectors of B are linearly independent.
A given set of column vectors of A forms a basis for
the column space of A if and only if the
corresponding column vectors of B form a basis for
the column space of B.
226
Theorems

Theorem 50

If a matrix R is in row echelon form, then the
row vectors with the leading 1’s (i.e., the
nonzero row vectors) form a basis for the row
space of R, and the column vectors with the
leading 1’s of the row vectors form a basis for
the column space of R.
227
Example

Find bases for the row and column spaces of
 1 3 4 2 5 4 
 2 6 9 1 8 2 

A
 2 6 9 1 9 7 


 1 3 4 2 5 4 
229
Example

Solution:

Reducing A to row-echelon form we obtain
 1 3 4 2 5 4 
 2 6 9 1 8 2 

A
 2 6 9 1 9 7 


 1 3 4 2 5 4 

Note about the
correspondence!
1 3
0 0
R
0 0

0 0
4 2 5 4 
1 3 2 6 
0 0 1 5

0 0 0 0
By Theorem 5.5.6 and 5.5.5(b), the row and column
spaces are
r1 = [1 -3 4 -2 5 4]
r2 = [0 0 1 3 -2 -6]
r3 = [0 0 0 0 1 5]
and
1
4
5
2
9
8
c1    , c3    , c5   
2
9
9
 
 
 
 1
 4
 5
230
Example (Basis for a Vector Space
Using Row Operations )

Find a basis for the space spanned by the vectors
v1= (1, -2, 0, 0, 3), v2 = (2, -5, -3, -2, 6),
v3 = (0, 5, 15, 10, 0), v4 = (2, 6, 18, 8, 6).
231
Example (Basis for a Vector Space
Using Row Operations )

Solution: (Write down the vectors as row vectors
first!)
1 2 0 0
 2 5 3 2

 0 5 15 10

 2 6 18 8
3
6 
0

6
1 2
0 1

0 0

0 0
0
3
1
0
0
2
1
0
3
0 
0

0
The nonzero row vectors in this matrix are
w1= (1, -2, 0, 0, 3), w2 = (0, 1, 3, 2, 0), w3 = (0, 0, 1, 1, 0)
 These vectors form a basis for the row space and
consequently form a basis for the subspace of R5
spanned by v1, v2, v3, and v4.

232
Remarks


Keeping in mind that A and R may have different
column spaces, we cannot find a basis for the
column space of A directly from the column
vectors of R.
However, it follows from Theorem 5.5.5b that if
we can find a set of column vectors of R that
forms a basis for the column space of R, then the
corresponding column vectors of A will form a
basis for the column space of A.
233
Remarks


In the previous example, the basis vectors
obtained for the column space of A consisted of
column vectors of A, but the basis vectors
obtained for the row space of A were not all
vectors of A.
Transpose of the matrix can be used to solve this
problem.
234
Example (Basis for the Row Space
of a Matrix )

Find a basis for the row space of
1 2 0 0
 2 5 3 2
A
 0 5 15 10

 2 6 18 8
3
6 
0

6
consisting entirely of row vectors from A.
235
Example (Basis for the Row Space
of a Matrix )

Solution:
 1 2 0 2
 2 5 5 6 


T
A   0 3 15 18


0

2
10
8


 3 6 0 6 

1
0

0

0
0
2 0
1 5
0 0
0 0
0 0
2 
10 
1 

0 
0 
The column space of AT are
1
2
2
 2 
 5
6
 
 
 
c1   0  , c 2   3 , and c4  18
 
 
 
0

2
 
 
8
 3 
 6 
 6 

Thus, the row space of A are
r1 = [1 -2 0 0 3]r2 = [2 -5 -3 -2 6]r3 = [2 6 18 8 6]
236
Example (Basis and Linear
Combinations )


(a) Find a subset of the vectors v1 = (1, -2, 0, 3), v2
= (2, -5, -3, 6), v3 = (0, 1, 3, 0), v4 = (2, -1, 4, -7),
v5 = (5, -8, 1, 2) that forms a basis for the space
spanned by these vectors.
(b) Express each vector not in the basis as a linear
combination of the basis vectors.
237
Example (Basis and Linear
Combinations )

Solution (a):
 1
 2

 0

 3

v1

2
5
3
6
0
1
3
0
2
1
4
7

v2
 
v3 v 4
5
 8
1 

2

v5
1
0

0

0
1
1
1

0
    
w1 w 2 w 3 w 4 w 5
0 2
1 1
0 0
0 0
0
0
1
0
Thus, {v1, v2, v4} is a basis for the column
space of the matrix.
238
Example

Solution (b):

We can express w3 as a linear combination of
w1 and w2, express w5 as a linear combination
of w1, w2, and w4 (Why?). By inspection, these
linear combination are
w3 = 2w1 – w2
w5 = w1 + w2 + w4
239
Example

We call these the dependency equations. The
corresponding relationships in the original
vectors are
v3 = 2v1 – v2
v5 = v1 + v2 + v4
240
Four Fundamental Matrix Spaces

Consider a matrix A and its transpose AT together, then
there are six vector spaces of interest:




row space of A, row space of AT
column space of A, column space of AT
null space of A, null space of AT
However, the fundamental matrix spaces associated
with A are


row space of A, column space of A
null space of A, null space of AT
241
Four Fundamental Matrix Spaces


If A is an mn matrix, then the row space of A and
nullspace of A are subspaces of Rn and the column
space of A and the nullspace of AT are subspace of Rm
What is the relationship between the dimensions of
these four vector spaces?
242
Dimension and Rank(秩)

Theorem 51


If A is any matrix, then the row space and column
space of A have the same dimension.
Definition

The common dimension of the row and column
space of a matrix A is called the rank of A and is
denoted by rank(A); the dimension of the nullspace
of a is called the nullity(零度) of A and is denoted
by nullity(A).
243
Example (Rank and Nullity)

Find the rank and nullity of the matrix
 1 2 0 4 5 3
 3 7 2 0 1 4 

A
 2 5 2 4 6 1 


 4 9 2 4 4 7 

Solution:

The reduced row-echelon form of A is
1
0

0

0

0 4 28 37 13
1 2 12 16 5 
0 0
0
0
0

0 0
0
0
0
Since there are two nonzero rows, the row space and
column space are both two-dimensional, so rank(A) = 2. 244
Example (Rank and Nullity)

The corresponding system of equations will be
x1 – 4x3 – 28x4 – 37x5 + 13x6 = 0
x2 – 2x3 – 12x4 – 16 x5+ 5 x6 = 0
245
Example (Rank and Nullity)

It follows that the general solution of the
system is
x1 = 4r + 28s + 37t – 13u,
x2 = 2r + 12s + 16t – 5u,
x3 = r, x4 = s, x5 = t, x6 = u
or  x  4 28 37 13
1
x 
 2  12  16 
 5 
2
 
     


 x3 
1   0   0 
 0 
   r   s t  u

x
0
1
0
0
4
 
     


 x5 
0  0   1 
 0 
 
     


 0   0   0 
 1 
 x6 

Thus, nullity(A) = 4.
246
Theorems

Theorem 52


If A is any matrix, then rank(A) = rank(AT).
Theorem 53 (Dimension Theorem for Matrices)

If A is a matrix with n columns, then rank(A) +
nullity(A) = n.
247
Theorems

Theorem 54

If A is an mn matrix, then:


rank(A) = Number of leading variables in the solution of
Ax = 0.
nullity(A) = Number of parameters in the general solution
of Ax = 0.
248
Example (Sum of Rank and
Nullity)

The matrix
 1 2 0 4 5 3
 3 7 2 0 1 4 

A
 2 5 2 4 6 1 


 4 9 2 4 4 7 

has 6 columns, so rank(A) + nullity(A) = 6
This is consistent with the previous example,
where we showed that
rank(A) = 2 and nullity(A) = 4
249
Example


Find the number of parameters in the
general solution of Ax = 0 if A is a 57
matrix of rank 3.
Solution:


nullity(A) = n – rank(A) = 7 – 3 = 4
Thus, there are four parameters.
250
Dimensions of Fundamental
Spaces

Suppose that A is an mn matrix of rank r,
then


AT is an nm matrix of rank r by Theorem 5.6.2
nullity(A) = n – r, nullity(AT) = m – r by Theorem
5.6.3
Fundamental Space Dimension
Row space of A
r
Column space of A r
Nullspace of A
n–r
Nullspace of AT
m–r
251
Maximum Value for Rank



If A is an mn matrix
 The row vectors lie in Rn and the column vectors lie
in Rm.
 The row space of A is at most n-dimensional and the
column space is at most m-dimensional.
Since the row and column space have the same
dimension (the rank A), we must conclude that if m  n,
then the rank of A is at most the smaller of the values of
m or n.
That is, rank(A)  min(m, n)
252
Theorems

Theorem 55 (The Consistency Theorem)

If Ax = b is a linear system of m equations in n
unknowns, then the following are equivalent.




Ax = b is consistent.
b is in the column space of A.
The coefficient matrix A and the augmented matrix [A | b] have
the same rank.
Theorem 56

If Ax = b is a linear system of m equations in n
unknowns, then the following are equivalent.



Ax = b is consistent for every m1 matrix b.
The column vectors of A span Rm.
rank(A) = m.
254
Overdetermined System



A linear system with more equations than
unknowns is called an overdetermined linear
system.
If Ax = b is an overdetermined linear system of
m equations in n unknowns (so that m > n), then
the column vectors of A cannot span Rm.
Thus, the overdetermined linear system Ax = b
cannot be consistent for every possible b.
255
Theorems

Theorem 57


If Ax = b is consistent linear system of m equations in
n unknowns, and if A has rank r, then the general
solution of the system contains n – r parameters.
Theorem 58

If A is an mn matrix, then the following are
equivalent.



Ax = 0 has only the trivial solution.
The column vectors of A are linearly independent.
Ax = b has at most one solution (0 or 1) for every m1
matrix b.
258
Theorem (Equivalent Statements)

If A is an mn matrix, and if TA : Rn  Rn is
multiplication by A, then the following are
equivalent:






A is invertible.
Ax = 0 has only the trivial solution.
The reduced row-echelon form of A is In.
A is expressible as a product of elementary
matrices.
Ax = b is consistent for every n1 matrix b.
Ax = b has exactly one solution for every n1
matrix b.
260
Theorem (Equivalent Statements)










The range of TA is Rn.
TA is one-to-one.
The column vectors of A are linearly independent.
The row vectors of A are linearly independent.
The column vectors of A span Rn.
The row vectors of A span Rn.
The column vectors of A form a basis for Rn.
The row vectors of A form a basis for Rn.
A has rank n.
A has nullity 0.
261