Chapter 07 - Diagonalization and Quadratic Forms PDF

Preview

Summary

Diagonalization and Quadratic Forms...

Description

CHAPTER

7

Diagonalization and Quadratic Forms CH APT E R CO N T E N T S

7.1 Orthogonal Matrices 401 7.2 Orthogonal Diagonalization 409 7.3 Quadratic Forms 417 7.4 Optimization Using Quadratic Forms 429 7.5 Hermitian, Unitary, and Normal Matrices 437

IN T RO DU CT IO N

In Section 5.2 we found conditions that guaranteed the diagonalizability of an n × n matrix, but we did not consider what class or classes of matrices might actually satisfy those conditions. In this chapter we will show that every symmetric matrix is diagonalizable. This is an extremely important result because many applications utilize it in some essential way.

7.1 Orthogonal Matrices In this section we will discuss the class of matrices whose inverses can be obtained by transposition. Such matrices occur in a variety of applications and arise as well as transition matrices when one orthonormal basis is changed to another.

Orthogonal Matrices Recall from Theorem 1.6.3 that if either product in (1) holds, then so does the other. Thus, A is orthogonal if either AAT = I or ATA = I .

We begin with the following definition. DEFINITION 1 A square matrix A is said to be orthogonal if its transpose is the same as its inverse, that is, if

A−1 = AT

or, equivalently, if

AAT = ATA = I

(1)

E X A M P L E 1 A 3 × 3 Orthogonal Matrix

The matrix

⎡

3 7

⎢ 6 A=⎢ ⎣− 7 2 7

is orthogonal since

⎡

3 7

⎢2 ATA = ⎢ ⎣7 6 7

− 76 3 7 2 7

⎤⎡

2 3 7 7 ⎢ ⎥ 6⎥⎢ 6 − 7⎦⎣ 7 3 2 −7 7

⎤

2 7 3 7 6 7

6 7 ⎥ 2⎥ 7⎦ − 73 2 7 3 7 6 7

⎤

6 7 ⎥ 2⎥ 7⎦ − 37

⎡

⎤

1

0

0

⎢ = ⎣0

1

0⎦

0

0

⎥

1

401

402

Chapter 7 Diagonalization and Quadratic Forms

E X A M P L E 2 Rotation and Reflection Matrices Are Orthogonal

Recall from Table 5 of Section 4.9 that the standard matrix for the counterclockwise rotation of R 2 through an angle θ is



cos θ A= sin θ

− sin θ cos θ



This matrix is orthogonal for all choices of θ since

ATA =



cos θ − sin θ

sin θ cos θ



cos θ sin θ

   1 0 − sin θ = cos θ 0 1

We leave it for you to verify that the reflection matrices in Tables 1 and 2 and the rotation matrices in Table 6 of Section 4.9 are all orthogonal.

Observe that for the orthogonal matrices in Examples 1 and 2, both the row vectors and the column vectors form orthonormal sets with respect to the Euclidean inner product. This is a consequence of the following theorem.

THEOREM 7.1.1 The following are equivalent for an n × n matrix A.

(a) A is orthogonal.

(b) The row vectors of A form an orthonormal set in R n with the Euclidean inner product. (c) The column vectors of A form an orthonormal set in R n with the Euclidean inner product.

Proof We will prove the equivalence of (a) and (b) and leave the equivalence of (a) and

(c) as an exercise. (a) ⇔ (b) Let ri be the i th row vector and cj the j th column vector of A. Since transposing a matrix converts its columns to rows and rows to columns, it follows that cjT = rj . Thus, it follows from the row-column rule [Formula (5) of Section 1.3] and the bottom form listed in Table 1 of Section 3.2 that

r1 c1T

r1 cT2

⎢ T ⎢ r2 c1 ⎢ AAT = ⎢ ⎢ .. ⎢ . ⎣

r2 cT2

⎡

rn c1T

... rn cT2

···

r1 cnT

⎤

⎡

⎤

r1 · r1

r1 · r2

r1 · rn

⎢ ⎢ r2 · r1 ⎢ ⎥=⎢ .. ⎥ ⎢ .. ⎢ . ⎥ ⎦ ⎣ . · · · rn cTn rn · r1

···

r2 · r2

···

r2 · rn ⎥ ⎥

···

r2 cnT ⎥ ⎥

⎥

... rn · r2

.. .

· · · rn · rn

It is evident from this formula that AAT = I if and only if

⎥ ⎥ ⎥ ⎥ ⎦

r1 · r1 = r2 · r2 = · · · = rn · rn = 1 WARNING Note that an or-

thogonal matrix has orthonormal rows and columns—not simply orthogonal rows and columns.

and ri · rj = 0 when i = j

which are true if and only if {r1 , r2 , . . . , rn } is an orthonormal set in R n . The following theorem lists four more fundamental properties of orthogonal matrices. The proofs are all straightforward and are left as exercises.

7.1 Orthogonal Matrices

403

THEOREM 7.1.2

(a) The transpose of an orthogonal matrix is orthogonal. (b) The inverse of an orthogonal matrix is orthogonal. (c) A product of orthogonal matrices is orthogonal. (d) If A is orthogonal, then det(A) = 1 or det(A) = −1.

E X A M P L E 3 det(A) = ±1 for an Orthogonal Matrix A

The matrix

A=

1 2 − √12



√

√1

2 √1 2



is orthogonal since its row (and column) vectors form orthonormal sets in R 2 with the Euclidean inner product. We leave it for you to verify that det(A) = 1 and that interchanging the rows produces an orthogonal matrix whose determinant is −1. Orthogonal Matrices as Linear Operators

We observed in Example 2 that the standard matrices for the basic reflection and rotation operators on R 2 and R 3 are orthogonal. The next theorem will explain why this is so. THEOREM 7.1.3 If

A is an n × n matrix, then the following are equivalent.

(a) A is orthogonal.

(b) Ax = x for all x in R n .

(c) Ax · Ay = x · y for all x and y in R n .

Proof We will prove the sequence of implications (a) (a) ⇒ (b) Assume that A is orthogonal, so that ATA

of Section 3.2 that

⇒ (b) ⇒ (c) ⇒ (a).

= I . It follows from Formula (26)

Ax = (Ax · Ax)1/2 = (x · ATAx)1/2 = (x · x)1/2 = x (b) ⇒ (c) Assume that Ax

= x for all x in R n . From Theorem 3.2.7 we have

Ax · Ay = 41 Ax + Ay2 − 14 Ax − Ay2 = 41 A(x + y)2 − 14 A(x − y)2 = 41 x + y2 − 14 x − y2 = x · y

(c) ⇒ (a) Assume that Ax · Ay

of Section 3.2 that

= x · y for all x and y in R n . It follows from Formula (26) x · y = x · ATAy

which can be rewritten as x · (ATAy − y) = 0 or as

x · (ATA − I )y = 0

Since this equation holds for all x in R n , it holds in particular if x = (ATA − I )y, so

(ATA − I )y · (ATA − I )y = 0

Thus, it follows from the positivity axiom for inner products that

(ATA − I )y = 0

404

Chapter 7 Diagonalization and Quadratic Forms

Since this equation is satisfied by every vector y in R n , it must be that ATA − I is the zero matrix (why?) and hence that ATA = I . Thus, A is orthogonal.

TA (u)

TA (v)

β v α

u

0 ||TA(u)|| = ||u||, TA (v)|| = ||v|| α = β, d (TA (u), TA (v)) = d(u, v)

Figure 7.1.1

Change of Orthonormal Basis

Theorem 7.1.3 has a useful geometric interpretation when considered from the viewpoint of matrix transformations: If A is an orthogonal matrix and TA : R n →R n is multiplication by A, then we will call TA an orthogonal operator on R n . It follows from parts (a) and (b) of Theorem 7.1.3 that the orthogonal operators on R n are precisely those operators that leave the lengths (norms) of vectors unchanged. However, as illustrated in Figure 7.1.1, this implies that orthogonal operators also leave angles and distances between vectors in R n unchanged since these can be expressed in terms of norms [see Definition 2 and Formula (20) of Section 3.2]. Orthonormal bases for inner product spaces are convenient because, as the following theorem shows, many familiar formulas hold for such bases. We leave the proof as an exercise. THEOREM 7.1.4 If

S is an orthonormal basis for an n-dimensional inner product space

V, and if then:

(u)S = (u1 , u 2 , . . . , un ) and (v)S = (v1 , v 2 , . . . , v n )

 u12 + u22 + · · · + u2n  (b) d(u, v) = (u1 − v1 )2 + (u2 − v2 )2 + · · · + (un − vn )2

(a) u =

(c) u, v = u1 v1 + u2 v2 + · · · + un vn

Remark Note that the three parts of Theorem 7.1.4 can be expressed as u = (u)S 

  d(u, v) = d (u)S , ( v)S

  u, v = (u)S , (v)S

where the norm, distance, and inner product on the left sides are relative to the inner product on V and on the right sides are relative to the Euclidean inner product on R n .

Transitions between orthonormal bases for an inner product space are of special importance in geometry and various applications. The following theorem, whose proof is deferred to the end of this section, is concerned with transitions of this type.

THEOREM 7.1.5 Let

V be a finite-dimensional inner product space. If P is the transition matrix from one orthonormal basis for V to another orthonormal basis for V, then P is an orthogonal matrix.

E X A M P L E 4 Rotation of Axes in 2-Space

In many problems a rectangular xy-coordinate system is given, and a new x ′y ′ -coordinate system is obtained by rotating the xy-system counterclockwise about the origin through an angle θ . When this is done, each point Q in the plane has two sets of coordinates— coordinates (x, y) relative to the xy-system and coordinates (x ′, y ′ ) relative to the x ′ y ′ system (Figure 7.1.2a). By introducing unit vectors u1 and u2 along the positive x - and y -axes and unit vectors u1′ and u2′ along the positive x ′ - and y ′ -axes, we can regard this rotation as a change

7.1 Orthogonal Matrices

405

from an old basis B = {u1 , u2 } to a new basis B ′ = {u1′, u′2 } (Figure 7.1.2b). Thus, the new coordinates (x ′, y ′ ) and the old coordinates (x, y) of a point Q will be related by

   ′ x −1 x = P y y′

(2)

where P is the transition from B ′ to B . To find P we must determine the coordinate matrices of the new basis vectors u1′ and u2′ relative to the old basis. As indicated in Figure 7.1.2c, the components of u1′ in the old basis are cos θ and sin θ , so

[u1′ ]B =



cos θ sin θ



Similarly, from Figure 7.1.2d we see that the components of u2′ in the old basis are cos(θ + π/2) = − sin θ and sin(θ + π/2) = cos θ , so

[u′2 ]B =

  − sin θ cos θ

Thus the transition matrix from B ′ to B is  cos θ P = sin θ



(3)

x cos θ + y sin θ

(5)

− sin θ cos θ

Observe that P is an orthogonal matrix, as expected, since B and B ′ are orthonormal bases. Thus   cos θ sin θ −1 T P =P = − sin θ cos θ so (2) yields    ′  cos θ sin θ x x = (4) − sin θ cos θ y y′ or, equivalently,

x′ = ′

y = −x sin θ + y cos θ

These are sometimes called the rotation equations for R 2 . y´

y

y´ (x, y) u´ (x´, y´) 2

Q

y´

u´1

x

x´

u1´ θ

θ

sin θ

π sin θ + 2

(

)

θ+

x

π 2

x´ θ

x

cos θ

u1

(a)

y

u´2

u2

x´ θ

y

(

π cos θ + 2

(b)

(c)

) (d )

Figure 7.1.2

E X A M P L E 5 Rotation of Axes in 2-Space

Use form (4) of the rotation equations for R 2 to find the new coordinates of the point Q(2, 1) if the coordinate axes of a rectangular coordinate system are rotated through an angle of θ = π/4. Solution Since

sin

π 4

= cos

π 4

1

=√

2

406

Chapter 7 Diagonalization and Quadratic Forms

the equation in (4) becomes

⎡ 1  ′ √ x 2 ⎣ ′ = 1 y √ −

√1

2

√1

2

2

⎤

  ⎦ x y

Thus, if the old coordinates of a point Q are (x, y ) = (2, −1), then

⎡ 1  ′ √ x 2 ⎣ = 1 y′ √ −

2

⎦

√1

2

2

′

⎤

√1

′

so the new coordinates of Q are (x , y ) =



2

−1





⎡

=⎣

3 √1 , − √ 2 2

1 √

2

− √3

2



.

⎤

⎦

Remark Observe that the coefficient matrix in (4) is the same as the standard matrix for the linear operator that rotates the vectors of R 2 through the angle −θ (see margin note for Table 5 of Section 4.9). This is to be expected since rotating the coordinate axes through the angle θ with the vectors of R 2 kept fixed has the same effect as rotating the vectors in R 2 through the angle −θ with the axes kept fixed. z

u3

E X A M P L E 6 Application to Rotation of Axes in 3-Space

z´

u´3 y´ u2´ y

u1 x

u2

u1´ θ x´

Figure 7.1.3

Suppose that a rectangular xyz -coordinate system is rotated around its z-axis counterclockwise (looking down the positive z-axis) through an angle θ (Figure 7.1.3). If we introduce unit vectors u1 , u2 , and u3 along the positive x -, y -, and z-axes and unit vectors u1′ , u2′ , and u′3 along the positive x ′ -, y ′ -, and z′ -axes, we can regard the rotation as a change from the old basis B = {u1 , u2 , u3 } to the new basis B ′ = {u1′ , u′2 , u3′ }. In light of Example 4, it should be evident that

⎡

⎡

⎤

⎤

cos θ − sin θ ⎥ ⎥ ′] = ⎢ ′] = ⎢ [u1 B ⎣sin θ ⎦ and [u2 B ⎣ cos θ⎦ 0 0

Moreover, since u′3 extends 1 unit up the positive z′ -axis,

⎡ ⎤ 0

⎢ ⎥ [u′3 ]B = ⎣ 0⎦ 1

′

It follows that the transition matrix⎡from B to B is cos θ − sin θ ⎢ cos θ P = ⎣ sin θ 0

0

′

and the transition matrix from B to B is ⎡ cos θ ⎢ P −1 = ⎣− sin θ 0

′

′

⎤

0 ⎥ 0⎦ 1

⎤

sin θ cos θ

0 ⎥ 0⎦

0

1

′

(verify). Thus, the new coordinates (x , y , z ) of a point Q can be computed from its old coordinates (x, y, z) by

⎡ ⎤ ⎡ cos θ sin θ x′ ⎢ ′⎥ ⎢ ⎣ y ⎦ = ⎣− sin θ cos θ z′ 0 0 O PT I O N A L

⎤⎡ ⎤

x 0 ⎥ ⎢ y⎥ 0⎦ ⎣ ⎦ 1

z

We conclude this section with an optional proof of Theorem 7.1.5.

7.1 Orthogonal Matrices

407

Proof of Theorem 7.1.5 Assume that

Recall that (u)S denotes a coordinate vector expressed in comma-delimited form whereas [u]S denotes a coordinate vector expressed in column form.

V is an n-dimensional inner product space and that P is the transition matrix from an orthonormal basis B ′ to an orthonormal basis B . We will denote the norm relative to the inner product on V by the symbol  V to distinguish it from the norm relative to the Euclidean inner product on R n , which we will denote by  . To prove that P is orthogonal, we will use Theorem 7.1.3 and show that P x = x for every vector x in R n . As a first step in this direction, recall from Theorem 7.1.4(a) that for any orthonormal basis for V the norm of any vector u in V is the same as the norm of its coordinate vector with respect to the Euclidean inner product, that is, uV = [u]B ′  = [u]B  or (6)

uV = [u]B ′  = P [u]B ′ 

Now let x be any vector in R n , and let u be the vector in V whose coordinate vector with respect to the basis B ′ is x, that is, [u]B ′ = x. Thus, from (6),

u = x = P x which proves that P is orthogonal.

Exercise Set 7.1 In each part of Exercises 1–4, determine whether the matrix is orthogonal, and if so find it inverse. 1. (a)

2. (a)



1

0

0

−1

1

0

0

1





(b)



(b)

0

1

⎢ 3. (a) ⎢ ⎣1

0

0 ⎥ ⎦

0

⎡1

2

√

√1

1 √

2

1 5 √2 5

√2

⎤

1 2

− 56

1 6

1 6

1 6

1 6

− 65

⎥ ⎥ ⎥ ⎥ −65 ⎥ ⎦ 1 6

1 6

⎡

2



8. Let TA : R 3 →R 3 be multiplication by the orthogonal matrix in Exercise 6. Find TA (x) for the vector x = (0, 1, 4), and confirm TA (x) = x relative to the Euclidean inner product on R 3 .



5 √1 5

1 √

6

− √26 1 √

2

1 2

1 2



− √1 2

2

√1

1 2

2

⎢ ⎢1 ⎢2 4. (a) ⎢1 ⎢ ⎣2

⎥

√1

0

√1

⎡ 1 − √2 ⎢ (b) ⎢ ⎣ 0

⎤

√1 2

⎡



6

1

0

0

⎢ ⎢0 ⎢ (b) ⎢ ⎢0 ⎣

1 √

3

1 √

− 21

3

1 √

0

3

1 √

3

1 √

3

1 √

3

⎤ ⎥ ⎥ ⎦

0

⎤

⎢ 5. A = ⎣−

4 5 9 25

0

12 25

3 5

4 5

− 35

⎤

12 ⎥ − 25 ⎦ 16 25

⎡

⎢ 6. A = ⎣

1 3 2 3

− 23

2 3 − 32

− 31

9. Are the standard matrices for the reflections in Tables 1 and 2 of Section 4.9 orthogonal? 10. Are the standard matrices for the orthogonal projections in Tables 3 and 4 of Section 4.9 orthogonal? 11. What conditions must a and b satisfy for the matrix

 a+b a−b

b−a b+a



⎥ to be orthogonal? ⎥ ⎥ 1 ⎥ 12. Under what conditions will a diagonal matrix be orthogonal? ⎦ 0 13. Let a rectangular x ′y ′ -coordinate system be obtained by ro-

0⎥

0 1 2

In Exercises 5–6, show that the matrix is orthogonal three ways: first by calculating ATA, then by using part (b) of Theorem 7.1.1, and then by using part (c) of Theorem 7.1.1.

⎡

7. Let TA : R 3 →R 3 be multiplication by the orthogonal matrix in Exercise 5. Find TA (x) for the vector x = (−2, 3, 5), and confirm that TA (x) = x relative to the Euclidean inner product on R 3 .

2⎤ 3 1⎥ 3⎦ 2 3

tating a rectangular xy-coordinate system counterclockwise through the angle θ = π/3.

(a) Find the x ′y ′ -coordinates of the point whose xy-coordinates are (−2, 6). (b) Find the xy-coordina...