04 Linear Algebra (Vector Space) - Paul Dwakins PDF

Preview

Summary

Download 04 Linear Algebra (Vector Space) - Paul Dwakins PDF

Description

LINEAR ALGEBRA Vector Spaces

Paul Dawkins

Linear Algebra

Table of Contents Preface............................................................................................................................................. ii Vector Spaces ................................................................................................................................. 3 Introduction ................................................................................................................................................ 3 Vector Spaces ............................................................................................................................................. 5 Subspaces ................................................................................................................................................. 15 Span .......................................................................................................................................................... 25 Linear Independence ................................................................................................................................ 34 Basis and Dimension ................................................................................................................................ 45 Change of Basis ........................................................................................................................................ 61 Fundamental Subspaces ........................................................................................................................... 74 Inner Product Spaces ................................................................................................................................ 85 Orthonormal Basis ................................................................................................................................... 93 Least Squares ......................................................................................................................................... 105 QR-Decomposition ................................................................................................................................ 113 Orthogonal Matrices ............................................................................................................................... 122

© 2007 Paul Dawkins

i

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

Preface Here are my online notes for my Linear Algebra course that I teach here at Lamar University. Despite the fact that these are my “class notes”, they should be accessible to anyone wanting to learn Linear Algebra or needing a refresher. These notes do assume that the reader has a good working knowledge of basic Algebra. This set of notes is fairly self contained but there is enough Algebra type problems (arithmetic and occasionally solving equations) that can show up that not having a good background in Algebra can cause the occasional problem. Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed. 1. Because I wanted to make this a fairly complete set of notes for anyone wanting to learn Linear Algebra I have included some material that I do not usually have time to cover in class and because this changes from semester to semester it is not noted here. You will need to find one of your fellow class mates to see if there is something in these notes that wasn’t covered in class. 2. In general I try to work problems in class that are different from my notes. However, with a Linear Algebra course while I can make up the problems off the top of my head there is no guarantee that they will work out nicely or the way I want them to. So, because of that my class work will tend to follow these notes fairly close as far as worked problems go. With that being said I will, on occasion, work problems off the top of my head when I can to provide more examples than just those in my notes. Also, I often don’t have time in class to work all of the problems in the notes and so you will find that some sections contain problems that weren’t worked in class due to time restrictions. 3. Sometimes questions in class will lead down paths that are not covered here. I try to anticipate as many of the questions as possible in writing these notes up, but the reality is that I can’t anticipate all the questions. Sometimes a very good question gets asked in class that leads to insights that I’ve not included here. You should always talk to someone who was in class on the day you missed and compare these notes to their notes and see what the differences are. 4. This is somewhat related to the previous three items, but is important enough to merit its own item. THESE NOTES ARE NOT A SUBSTITUTE FOR ATTENDING CLASS!! Using these notes as a substitute for class is liable to get you in trouble. As already noted not everything in these notes is covered in class and often material or insights not in these notes is covered in class.

© 2007 Paul Dawkins

ii

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

Vector Spaces Introduction In the previous chapter we looked at vectors in Euclidean n-space and while in ¡ 2 and ¡ 3 we thought of vectors as directed line segments. A vector however, is a much more general concept and it doesn’t necessarily have to represent a directed line segment in ¡ 2 or ¡ 3 . Nor does a vector have to represent the vectors we looked at in ¡ n . As we’ll soon see a vector can be a matrix or a function and that’s only a couple of possibilities for vectors. With all that said a good many of our examples will be examples from ¡ n since that is a setting that most people are familiar with and/or can visualize. We will however try to include the occasional example that does not lie in ¡ n . The main idea of study in this chapter is that of a vector space. A vector space is nothing more than a collection of vectors (whatever those now are…) that satisfies a set of axioms. Once we get the general definition of a vector and a vector space out of the way we’ll look at many of the important ideas that come with vector spaces. Towards the end of the chapter we’ll take a quick look at inner product spaces. Here is a listing of the topics in this chapter. Vector Spaces – In this section we’ll formally define vectors and vector spaces. Subspaces – Here we will be looking at vector spaces that live inside of other vector spaces. Span – The concept of the span of a set of vectors will be investigated in this section. Linear Independence – Here we will take a look at what it means for a set of vectors to be linearly independent or linearly dependent. Basis and Dimension – We’ll be looking at the idea of a set of basis vectors and the dimension of a vector space. Change of Basis – In this section we will see how to change the set of basis vectors for a vector space. Fundamental Subspaces – Here we will take a look at some of the fundamental subspaces of a matrix, including the row space, column space and null space. Inner Product Spaces – We will be looking at a special kind of vector spaces in this section as well as define the inner product. Orthonormal Basis – In this section we will develop and use the Gram-Schmidt process for constructing an orthogonal/orthonormal basis for an inner product space. Least Squares – In this section we’ll take a look at an application of some of the ideas that we will be discussing in this chapter. © 2007 Paul Dawkins

3

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

QR-Decomposition – Here we will take a look at the QR-Decomposition for a matrix and how it can be used in the least squares process. Orthogonal Matrices – We will take a look at a special kind of matrix, the orthogonal matrix, in this section.

© 2007 Paul Dawkins

4

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

Vector Spaces As noted in the introduction to this chapter vectors do not have to represent directed line segments in space. When we first start looking at many of the concepts of a vector space we usually start with the directed line segment idea and their natural extension to vectors in ¡ n because it is something that most people can visualize and get their hands on. So, the first thing that we need to do in this chapter is to define just what a vector space is and just what vectors really are. However, before we actually do that we should point out that because most people can visualize directed line segments most of our examples in these notes will revolve around vectors in ¡n . We will try to always include an example or two with vectors that aren’t in ¡ n just to make sure that we don’t forget that vectors are more general objects, but the reality is that most of the examples will be in ¡ n . So, with all that out of the way let’s go ahead and get the definition of a vector and a vector space out of the way.

Definition 1 Let V be a set on which addition and scalar multiplication are defined (this means that if u and v are objects in V and c is a scalar then we’ve defined u + v and cu in some way). If the following axioms are true for all objects u, v, and w in V and all scalars c and k then V is called a vector space and the objects in V are called vectors. (a) u + v is in V – This is called closed under addition. (b) cu is in V – This is called closed under scalar multiplication. (c) u + v = v + u (d) u + ( v + w ) = ( u + v ) + w (e) There is a special object in V, denoted 0 and called the zero vector, such that for all u in V we have u + 0 = 0 + u = u . (f) For every u in V there is another object in V, denoted -u and called the negative of u, such that u - u = u + ( -u ) = 0 . (g) c (u + v ) = cu + cv (h) ( c + k ) u = c u + k u (i) c ( ku) = ( ck ) u (j) 1u = u We should make a couple of comments about these axioms at this point. First, do not get too locked into the “standard” ways of defining addition and scalar multiplication. For the most part we will be doing addition and scalar multiplication in a fairly standard way, but there will be the occasional example where we won’t. In order for something to be a vector space it simply must have an addition and scalar multiplication that meets the above axioms and it doesn’t matter how strange the addition or scalar multiplication might be. Next, the first two axioms may seem a little strange at first glance. It might seem like these two will be trivially true for any definition of addition or scalar multiplication, however, we will see at least one example in this section of a set that is not closed under a particular scalar multiplication. © 2007 Paul Dawkins

5

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

Finally, with the exception of the first two these axioms should all seem familiar to you. All of these axioms were in one of the theorems from the discussion on vectors and/or Euclidean nspace in the previous chapter. However, in this case they aren’t properties, they are axioms. What that means is that they aren’t to be proven. Axioms are simply the rules under which we’re going to operate when we work with vector spaces. Given a definition of addition and scalar multiplication we’ll simply need to verify that the above axioms are satisfied by our definitions. We should also make a quick comment about the scalars that we’ll be using here. To this point, and in all the examples we’ll be looking at in the future, the scalars are real numbers. However, they don’t have to be real numbers. They could be complex numbers. When we restrict the scalars to real numbers we generally call the vector space a real vector space and when we allow the scalars to be complex numbers we generally call the vector space a complex vector space. We will be working exclusively with real vector spaces and from this point on when we see vector space it is to be understood that we mean a real vector space. We should now look at some examples of vector spaces and at least a couple of examples of sets that aren’t vector spaces. Some of these will be fairly standard vector spaces while others may seem a little strange at first but are fairly important to other areas of mathematics.

Example 1 If n is any positive integer then the set V = ¡n with the standard addition and scalar multiplication as defined in the Euclidean n-space section is a vector space. Technically we should show that the axioms are all met here, however that was done in Theorem 1 from the Euclidean n-space section and so we won’t do that for this example. Note that from this point on when we refer to the standard vector addition and standard vector scalar multiplication we are referring to that we defined in the Euclidean n-space section.

Example 2 The set V = ¡2 with the standard vector addition and scalar multiplication defined as,

c (u1 , u 2 ) = ( u1 , cu 2 )

is NOT a vector space. Showing that something is not a vector space can be tricky because it’s completely possible that only one of the axioms fails. In this case because we’re dealing with the standard addition all the axioms involving the addition of objects from V (a, c, d, e, and f) will be valid. Also, in this case of all the axioms involving the scalar multiplication (b, g, h, i, and j), only (h) is not valid. We’ll show this in a bit, but the point needs to be made here that only one of the axioms will fail in this case and that is enough for this set under this definition of addition and multiplication to not be a vector space. First we should at least show that the set meets axiom (b) and this is easy enough to show, in that we can see that the result of the scalar multiplication is again a point in ¡ 2 and so the set is closed under scalar multiplication. Again, do not get used to this happening. We will see at least one example later in this section of a set that is not closed under scalar multiplication as we’ll define it there.

© 2007 Paul Dawkins

6

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

Now, to show that (h) is not valid we’ll need to compute both sides of the equality and show that they aren’t equal.

( c + k)u = ( c + k )( u1 , u2 ) = ( u1 , ( c + k ) u2 ) = ( u1, cu2 + ku2 ) cu + ku = c (u1 , u 2 ) + k (u1 , u 2 ) = (u1 , cu 2 ) + (u1 , ku 2 ) = ( 2u 1 , cu 2 + ku 2 )

So, we can see that ( c + k ) u ¹ cu + ku because the first components are not the same. This means that axiom (h) is not valid for this definition of scalar multiplication. We’ll not verify that the remaining scalar multiplication axioms are valid for this definition of scalar multiplication. We’ll leave those to you. All you need to do is compute both sides of the equal sign and show that you get the same thing on each side.

Example 3 The set V = ¡3 with the standard vector addition and scalar multiplication defined as,

c ( u1 , u 2 , u3 ) = ( 0, 0, cu 3 )

is NOT a vector space. Again, there is a single axiom that fails in this case. We’ll leave it to you to verify that the others hold. In this case it is the last axiom, (j), that fails as the following work shows.

1u =1 (u 1, u 2, u 3 ) = ( 0, 0, (1) u 3 ) = ( 0, 0, u 3 ) ¹ (u 1, u 2, u 3 ) = u

Example 4 The set V = ¡2 with the standard scalar multiplication and addition defined as, ( u1 , u2 ) + ( v1 , v2 ) = ( u1 + 2v1 , u2 + v2 ) Is NOT a vector space. To see that this is not a vector space let’s take a look at the axiom (c).

u + v = ( u1 , u 2 ) + ( v1 , v 2 ) = ( u1 + 2v1 , u 2 + v 2 ) v + u = ( v1, v 2 ) + (u1, u 2 ) = ( v1 + 2u1, v 2 + u 2 )

So, because only the first component of the second point listed gets multiplied by 2 we can see that u + v ¹ v + u and so this is not a vector space. You should go through the other axioms and determine if they are valid or not for the practice. So, we’ve now seen three examples of sets of the form V = ¡ n that are NOT vector spaces so hopefully it is clear that there are sets out there that aren’t vector spaces. In each case we had to change the definition of scalar multiplication or addition to make the set fail to be a vector space. However, don’t read too much into that. It is possible for a set under the standard scalar multiplication and addition to fail to be a vector space as we’ll see in a bit. Likewise, it’s possible for a set of this form to have a non-standard scalar multiplication and/or addition and still be a vector space. In fact, let’s take a look at the following example. This is probably going to be the only example that we’re going to go through and do in excruciating detail in this section. We’re doing this for two reasons. First, you really should see all the detail that needs to go into actually showing that a set along with a definition of addition and scalar multiplication is a vector space. Second, our © 2007 Paul Dawkins

7

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

definitions are NOT going to be standard here and it would be easy to get confused with the details if you had to go through them on your own.

Example 5 Suppose that the set V is the set of positive real numbers (i.e. x > 0 ) with addition and scalar multiplication defined as follows,

x + y = xy

cx = x c

This set under this addition and scalar multiplication is a vector space. First notice that we’re taking V to be only a portion of ¡ . If we took it to be all of ¡ we would not have a vector space. Next, do not get excited about the definitions of “addition” and “scalar multiplication” here. Even though they are not addition and scalar multiplication as we think of them we are still going to call them the addition and scalar multiplication operations for this vector space. Okay, let’s go through each of the axioms and verify that they are valid. First let’s take a look at the closure axioms, (a) and (b). Since by x and y are positive numbers their product xy is a positive real number and so the V is closed under addition. Since x is c positive then for any c x is a positive real number and so V is closed under scalar multiplication. Next we’ll verify (c). We’ll do this one with some detail pointing out how we do each step. First assume that x and y are any two elements of V (i.e. they are two positive real numbers).

We’ll now verify (d). Again, we’ll make it clear how we’re going about each step with this one. Assume that x, y, and z are any three elements of V.

Next we need to find the zero vector, 0, and we need to be careful here. We use 0 to denote the zero vector but it does NOT have to be the number zero. In fact in this case it can’t be zero if for no other reason than the fact that the number zero isn’t in the set V ! We need to find an element that is in V so that under our definition of addition we have,

x+ 0 = 0+ x = x It looks like we should define the “zero vector” in this case as : 0=1. In other words the zero vector for this set will be the number 1! Let’s see how that works and remember that our © 2007 Paul Dawkins

8

http://tutorial.math.lamar.edu/terms.aspx

Linear Algebra

“addition” here is really multiplication and remember to substitute the number 1 in for 0. If x is any element of V,

x + 0 = x ×1 = x

0 + x = 1× x = x

&

Sure enough that does what we want it to do. We next need to define the negative, - x , for each element x that is in V. As with the zero vector to not confuse - x with “minus x”, this is just the notation we use to denote the negative of x. In our case we need an element of V (so it can’t be minus x since that isn’t in V) such that

x + (- x) = 0

and remember that 0=1 in our case! Given an x in V we know that x is strictly positive and so positive (s...