Chapter 1 Lecture Note MA1511 PDF

Preview

Summary

MA1513 Chapter 1: (Linear Algebra) Matrix and System 1 Systems of Linear Equations (P) 1 Solving System of Linear Equations (P) 1 Gaussian Elimination (P) 1 Matrices (P) 1 Matrix Inverses I (P) 1 Matrix Inverses II (P) 1 Determinants of Matrices (P) 1 Ranks of Matrices (P) 1 Systems of Linear Equati...

Description

MA1513 Chapter 1: (Linear Algebra) Matrix and System 1.1 Systems of Linear Equations (P.1) 1.2 Solving System of Linear Equations (P.7) 1.3 Gaussian Elimination (P.14) 1.4 Matrices (P.18) 1.5 Matrix Inverses I (P.22) 1.6 Matrix Inverses II (P.25) 1.7 Determinants of Matrices (P.28) 1.8 Ranks of Matrices (P.33) 1.1 Systems of Linear Equations In this section, we introduce the most fundamental object in Linear Algebra – system of linear equations, which play an important role in the mathematical modelling of many engineering problems and other real life applications. Linear Equations (slide 2) In general, a linear equation in n variables is given by this algebraic expression:

a1x1 + a2x2 + ··· + anxn = b Here x1 to xn denote the variables, while a1 to an, as well as b represent some fixed numbers. Specifically, a1 to an are called the coefficients corresponding to the variables, and b is called the constant term. Here are some concrete examples. a) x + 3y = 7 b) y = x – 0.5z + 4.5 c)

x1 + x2 + ··· + xn = 1

d) x1 + 2x2 + 2x3 + x4 = x5 (a) is a linear equation in two variables x and y. This represents a line in the xy plane. (b) is a linear equation in three variables x, y, z. Notice the x and z terms are on the right side of the equation. In any equation, we are free to move the various terms between the left and right side. But the standard form is to place all the variables on the left, and the constant term on the right as such: -x + y + 0.5z = 4.5. In (c), instead of representing the variables using regular letters like x, y, z, we use the indexed notation x1, x2, …, xn. This is common when there are many variables in the equation. (d) also uses indexed notation. There are 5 variables x1 to x5. Again, we can bring the variable x5 to the left side to get the standard form. The key feature of a linear equation is that different variables in the equation can only be combined using addition or subtraction.

1

Non-linear Equations (slide 3) Here are some equations that are not linear. a) xy = 2 b) sin(θ) + cos (φ) = 0.2 c) x12 + x22+ ··· + xn2 = 1 d) x = ey (a) is not linear because it involves the multiplication of the two variables x and y. (b) is not linear in the variables θ and φ, as the sine and cosine functions are being applied to the variables. (c) involves the square of each variable x1 to xn, and hence is not linear. (d) is not linear because the exponential function is applied to the variable y. Geometrical Representation (slide 4) Let us look at the geometrical interpretation of linear equations to help us better visualize these algebraic objects. We will consider them by cases in term of the number of variables. If a linear equation has two variables x and y such as ax + by = c, then it represents a line in the xyplane.

(Note that in this course, when we use the word “line”, we always refer to straight line, rather than a line that curves or bends.) If a linear equation has three variables x, y and z such as ax + by +cz = d, then it does not represent a line in the xyz-space. Rather, it represents a flat plane in the 3D space.

A linear equation always represent objects that are straight or flat. Furthermore, such an equation always represent an object that is “one dimension smaller” than that of the space it is sitting in. If a linear equation ax + by + cz + dw = e has four variables (x, y, z and w), what kind of geometrical object does it represent? The answer is that it represents none. This is because with 4 variables, it means we need a 4dimensional space to hold the three dimensional object. This is something beyond visualization. Nevertheless, linear equations with more than 3 variables still have many real life applications in math modelling.

2

System of Linear Equations (slide 5) We are now ready to put a few linear equations to form a system of linear equations. ฀฀ ฀฀ + ฀฀12 ฀฀2 + … + ฀฀1฀฀ ฀฀฀ ฀ = ฀฀1 ⎧ 11 1 ฀฀21 ฀฀1 + ฀฀22 ฀฀2 + … + ฀฀2฀฀ ฀฀฀ ฀ = ฀฀2 ⋮ ⎨ ⋮ ⎩ ฀฀฀฀1 ฀฀1 + ฀฀฀฀2 ฀฀2 + … + ฀฀฀฀฀฀ ฀฀฀ ฀ = ฀฀฀฀ The equations must have the same set of variables in common. We will also call such a system simply as a linear system. From the indexed notation, we see that there are m linear equations in the system, and n common variables x1 to xn. Again, we denote the coefficients and constant terms by the a’s and the b’s respectively. You should note that, in order to better track the coefficients in the system, we have used a “double indexed” notation. For example, a12 refers to the coefficient of x2 in the 1st equation, and so on. Solutions of Linear System (slide 6) When we have a linear system of equations, one of the problems we would like to solve is to find solutions to the system. Specifically, for a system with n variables say x1 to xn, we want to find n (real) numbers, say s1, s2, .., sn that satisfy all the equations simultaneously. When this happen, we say s1 to sn is a solution of the system and write it in the following way to specify which number correspond to which variable:

x1 = s1, x2 = s2, …, xn = sn This s known as a particular solution of the linear system. Example: Linear System with more than 1 solution (slide 7)  4x1 − x2 + 3 x3 = −1  3x  1 + x2 + 9 x3 = −4

x1 = 1, x2 = 2, x3 = –1 is a (particular) solution, since it satisfies both equations in the system. x1 = 1, x2 = 8, x3 = 1 is not a solution, since it satisfies the first equation but not the second one. A linear system need not have just one solution. You can check that x1 = 0, x2 = -1/4, x3 = –5/12. is another particular solution of the system. In fact, we shall see later that when a system has more than 1 solution, it will have infinitely many solutions.

Example: Linear System with no solution (slide 8) With the constraint of each linear equation in a system, it is highly possible for a random system to have no solution at all.  x + y = 4  2x 2 y = 6 + 

This system has no solution. Observe the second equation can be simplified as x + y = 3. Then we see that the first equation requires the sum of the two variables to be 4, while the second equation requires the sum to be 3. In other words, the two equations are contradicting each other, or not consistent with each other, hence it is impossible to find a solution. In general, when we have three or more equations in a linear system, it may not be easy to tell whether the equations are inconsistent by observation. We will see later how to determine the consistency of a general linear system. 3

Number of Solutions of Linear Systems (slide 9) Here is an important fact about the number of solutions that any linear system can have. There are only three possibilities: • no solution • exactly one solution • infinitely many solutions For the first case, we call a system with no solution as an inconsistent system, as there are inconsistencies among the equations in the system. For the second and third cases, the system has at least one solution, and we say the system is consistent. In our earlier example  4x1 − x 2 + 3 x 3 = −1  3x + x + 9 x = −4 2 3  1

we have found two solutions. Since the system cannot have just two solutions, we conclude that it must belong to the third case, i.e. it has infinitely many solutions, even though we have not explicitly find them. We shall see later how to find all the solutions for the infinite case.

2 by 2 Linear Systems (slide 10) The reason that there are only three possible cases for the number of solutions for any system is because it is linear. The 2 x 2 system with two equations in two variables will provide some insight to this peculiar property.  a1x + b1y = c 1 ax+by =c 2 2  2

( l 1) ( l2)

Geometrically the two equations represent two lines in the xy-plane. Let’s called them ฀฀ 1 and ฀฀2. There are three possible relative positions for the two lines:

In the first diagram, the two lines ฀฀1 and ฀฀2 are parallel, so they have no intersection. This means the system has no solution.

In the second diagram, ฀฀1 and ฀฀2 are not parallel, so they will intersect at exactly one point. This corresponds to exactly one solution for the system.

In the third diagram, the two lines overlap, so ฀฀ 1 and ฀฀2 represent the same line. This means every point on the line is part of the intersection. Hence the system has infinitely many solutions. Homogeneous Linear Systems (slide 11) ฀฀11 ฀฀1 ฀฀ ฀฀ � 21 1 ⋮ ฀฀฀฀1 ฀฀1

฀฀12 ฀฀2 ฀฀22 ฀฀2

We say a system is homogeneous if the system looks like this: + + +

฀฀฀฀2 ฀฀2

+ +

… …

+ +

+

…

+

฀฀1฀฀ ฀฀฀ ฀฀2฀฀ ฀฀฀

฀฀฀฀฀฀ ฀฀฀

฀ ฀

฀

= =

⋮ =

0 0 0

Notice that it looks almost like an ordinary linear system, except that the constant terms are all 0.

4

On the other hand, if some of the constant terms are not zero, we say the system is nonhomogeneous. In the next two segments, we discuss some features of the solutions of homogeneous systems.

Trivial Solution of Homogeneous System (slide 12) There is an obvious solution to the homogeneous system. When we substitute all the variables with 0, they clearly satisfy all the equations, and hence x1 = 0, x2 = 0, …, xn = 0 is a solution of the system. We call this the trivial solution of the homogeneous system. In particular, a homogeneous system is always consistent regardless of the coefficients. It is possible for a homogeneous system to have only the trivial solution, or it might have other additional solutions. These additional solutions are called non-trivial solutions. Here’s an example of a homogeneous system that has more than one solution.  x1 − x2 + x3 = 0   x1 + x2 + 3 x3 = 0

First of all, it has the trivial solution x1 = 0, x2 = 0, x3 = 0. We can also easily check that x1 = 2, x2 = 1, x3 = -1 is another solution. So this is a non-trivial solution to the system, and we know this implies the system in fact has infinitely many solutions.

Number of Solutions of Homogeneous Systems (slide 13) Here are two properties about the number of solutions of homogeneous systems. 1. A homogeneous system of linear equations has either only the trivial solution or infinitely many solutions in addition to the trivial solution. 2. A homogeneous system of linear equations with more variables than equations has infinitely many solutions. Let’s give two examples:

The homogeneous system on the left has 2 equations and 3 variables; and the system on the right has 3 equations and 4 variables. Regardless of what their coefficients are, we can conclude that both systems have infinitely many solutions.

Trivial Vs Non-trivial Solutions (slide 14) Let’s illustrate using the 2 x 3 homogeneous system, with 2 equations and 3 variables.  a1x + b1y + c1z = 0 (P1 )   a2 x + b2 y + c2 z = 0 (P2 )

Recall these two equations represent two planes in the xyz-space. The constant term being zero, means the planes contain the origin. So the two planes intersect at least the origin, which correspond to the trivial solution. But the property in the previous segment tells us that the system has more than just the trivial solution. Let’s look at the possible positions of the two planes.

5

First possibility (left diagram) is the two planes intersect. It is not hard for us to see that they must intersect at a line, and hence has infinitely many solutions. Namely the trivial solution at the origin, and all the points along the intersecting line, representing the non-trivial solutions. The second possibility (right diagram) is when the two planes overlapped each other completely, and hence there are again infinitely many solutions. This time we have the trivial solution at the origin, and infinitely many other points on the plane representing the non-trivial solutions.

6

1.2 Solving System of Linear Equations When we are given a linear system, how do we determine whether it has a solution? If it does, how do we find all the solutions in a systematically way? This section provides the first step towards answering these questions.

Augmented Matrix (slide 2) ฀฀11 ฀฀1 ฀฀ ฀฀ � 21 1 ⋮ ฀฀฀฀1 ฀฀1

฀฀12 ฀฀2 ฀฀22 ฀฀2

฀฀1฀฀ ฀฀฀ ฀฀2฀฀ ฀฀฀

= ฀฀1 = ฀฀2 ⋮ = ฀฀฀฀

First, we come up with a slightly simpler notation to represent a linear system. + + +

฀฀฀฀2 ฀฀2

+ +

… …

+ +

+

…

+

฀฀฀฀฀฀ ฀฀฀

฀

฀

฀

We have a system with m equations and n variables. We take all the coefficients and constant terms, and arrange them in a rectangular array as shown. ฀฀11 ฀฀12 … ฀฀1฀฀ ฀฀1 ฀฀21 ฀฀22 … ฀฀2฀฀ ฀฀2 � � � ⋮ ⋮ ⋮ ⋮ ฀฀฀฀1 ฀฀฀฀2 … ฀฀฀฀฀฀ ฀฀฀฀ This array has m horizontal rows, corresponding to the m equations of the system, and n+1 vertical columns, corresponding to the n variables and the constant terms. Note that, to keep the notation simple, the variables, the plus and equal signs are not displayed in the array. Instead, a vertical line is drawn to separate the coefficients and the constant terms. We call this the augmented matrix of the linear system.

Elementary Row Operations (slide 3) What we are going to do with the augmented matrix is to reduce it to a simpler form, so that it becomes easier to solve the linear system. The way to reduce the augmented matrix is by performing some operations on the rows of the matrix. There are three types of row operations, namely • • •

Multiplying a row by a nonzero constant; Interchanging two rows; Adding a multiple of a row to another row.

These are the three basic operations on the rows of a matrix, which we called the elementary row operations.

Example: Elementary Row Operations (slide 4) For the first type of row operation, multiply the first row of the left augmented matrix by the number 3, we will get the augmented matrix on the right.  1 1 2 9  2 4 3 1 −    3 6 − 5 0  

 3 3 6 27    2 4 −3 1   3 6 −5 0   

Notice that all the entries in the first row get multiplied by 3. For the second type of row operation, we interchange the second and third row of the left augmented matrix to get the right. 7

 1 1 2 9    2 4 − 3 1  3 6 − 5 0  

1 1 2 9    3 6 −5 0  2 4 −3 1   

For the third type of row operation, we want to add 2 times of the first row to the second row. First, we multiply the first row of the left augmented matrix by 2 to get (2 2 4 | 18). Then we add this to the second row of the same matrix, by adding the corresponding entries. The resulting matrix looks like this.  1 1 2 9    2 4 −3 1  3 6 −5 0  

1 1 2 9     4 6 1 19  3 6 −5 0   

Note that in the resulting matrix, only the second row is changed; the first row will remain the same as the original matrix.

Notation of Elementary Row Operations (slide 5) Let us denote the rows of an augmented matrix to be operated on using the indexed notation Ri. So R1 represents the first row, R3 represents the third row and so on. With that, we can now represent the three row operations as follow: First type: cRi (multiplying the i th row by the constant c)

Second type: Ri ↔ Rj (interchanging the positions of the ith and jth rows) Third type: Ri + cRj (adding c times the jth row to the ith row)

In the last notation, it means the ith row will be changed, but the jth row remains the same after the operation.

Example: Solving Linear System (slide 6-9) We shall now elaborate how to solve a linear system using the augmented matrix and the elementary row operations. Do take note of the action taken on the left hand side on the system, and the corresponding row operations being performed on the right side.

In the first step above, we add -2 times of equation 1 to equation 2, and change equation 2 to equation 4 as shown. Correspondingly, the second row of the augmented matrix is changed accordingly by adding -2 times of first row of the augmented matrix to the second row as shown in the diagram.

8

In this next step, we change equation 3 to equation 5. This correspond to the row operation of adding -3 times of the first row of the matrix to the third row.

Next, we change equation 5 to equation 6. This corresponds to the row operation of adding 6/4 times of the second row of the matrix to the third row. Let’s summarize how the original augmented matrix has been transformed under the operations: 1 1 3   2 −2 2 3 9 0 

0  4 3 

1 1 3  0 −4 −4 3 9 0 

0  4 3 

1 1 3  0 − 4 − 4 0 6 − 9

0  4 3 

1 1 3  − − 0 4 4  0 0 − 15

0  4 9 

Observe that the augmented matrix now has a “staircase” of 0 at the bottom left corner of the array. This is the simplified form of the augmented matrix that we are looking for. It is known as the row echelon form (which will be defined more precisely later).

If we look at the corresponding linear system, we notice the number of variables in each equation decreases as we go down the row. This will facilitate us to solve the equations from bottom up, which we call back substitution.

First, from (6), we can easily solve z = -3/5. Substituting this value of z into (4), we get -4y - 4(-3/5) = 4 which gives y = -2/5. Finally, substituting the values of y and z into (1), we get x -2/5 -9/5 = 0, which gives x = -11/5. These three numbers give us a solution of the linear system. In fact, from the process, we see that this is the only solution of the original linear system.

9

    

Row Echelon Form (R.E.F.) (slide 10) Let us take a closer look at the row echelon form of an augmented matrix, and introduce some terminologies that will be useful. First of all, in a row echelon form, any zero rows must be below all the nonzero rows. A zero row is one whereby all the entries are 0: (0 0 … 0 | 0); and a nonzero row is one with some nonzero entries. For example, (0 0 3 0 1 | 0) and (0 0 0 0 0 | 2) are examples of non-zero rows. If we go from left to right on each nonzero row, the first nonzero entry is called the leading entry. So the leading entry of (0 0 3 0 1 | 0) is 3, and the leading entry of (0 0 0 0 0 | 2) is 2.

In the diagram below, the symbol ⊗ represents the leading entry of each non-zero row. (Note that entries * on the right of the leading entry can be either zero or non-zero.) 0 ⊗ ∗ ∗ ∗ ∗ ∗ ∗ 0 0 0 ⊗ ∗ ∗ ∗ ∗ ⎛ ⎞ 0 0 0 0 0 ⊗ ∗ �∗ ⎜ ⎟ ⎜0 0 0 0 0 0 ⊗ �∗⎟ 0 0 0 0 0 0 0 0 ⎝0 0 0 0 0 0 0 0⎠

The most important feature of a row echelon form is that the number of zeros on the left of the leading entries is strictly increasing as we go down the row, creating a staircase of zeros at the bottom left of the matrix. Note that the width of each stair need not be the same. Now let’s look at columns of the augmented matrix. We are interested in those columns that contain the leading entries. We call these columns the pivot columns.

0 ⊗ ∗ ∗ ∗ ∗ ∗ ∗ 0 0 0 ⊗ ∗ ∗ ∗ ∗ ⎛ ⎞ 0 0 0 0 0 ⊗ ∗ �∗ ⎜ ⎟ ⎜0 0 0 0 0 0 ⊗ �∗⎟ 0 0 0 0 0 0 0 0 ⎝0 0 0 0 0 0 0 0⎠ In the above augmented matrix, the pivot columns are the 2nd, 4th, 6th and 7th columns. The remaining columns in the matrix are called the non-pivot columns.

The pivot and non-pivot columns in the row echelon form play an important role in determining the number of solutions of a linear system.

Example: System with Infinitely Many Solutions (slide 11)

This is a system that is already reduced. In other words, its augmented matrix is in row echelon form. This system has 4 variables and 3 equations. In other words, there are more variables than equations. We need to set s...