Notes on Quadratic Forms PDF

Preview

Summary

Notes on Quadratic Forms...

Description

Economics 211 Notes on Quadratic Forms

. Quadratic forms and their properties are used to state the second-order conditions for functions of two or more variables. If A is an n by n matrix and x is an n by 1 column vector then the quadratic form of A and x is QF = x⊤ Ax. Note that QF is a one by one matrix, that is, a scalar. So  ⊤ QF = x⊤ Ax = x⊤ Ax = x⊤ A⊤ x.

⊤ This shows that  the⊤QF  of A equals the QF of A . So for QF purposes, if A is not symmetric, B = A + A /2 will be and the QF of A must equal the QF of B . For example, if A is 3 by 3 then

QF =



x1 x2 x3



  a11 a12 a13 x1  a21 a22 a23   x2  a31 a32 a33 x3 

= a11x21 + a22x22 + a33x23 + (a12 + a21 ) x1 x2 + (a13 + a31) x1 x3 + (a23 + a32 ) x2 x3 = b11x21 + b22x22 + b33x32 + 2b12 x1 x2 + 2b13x1 x3 + 2b23 x2 x3 , where bij = (aij + aj i ) /2. So if we are calculating the QF of A and A is not already symmetric we will work with QF of B which equals the QF of A and B is symmetric. A is positive definite if QF > 0 for all x vectors in which at least one element is nonzero. (Clearly if all elements of x were zero the QF would have to equal zero.) In particular, this means that every element of A’s main diagonal must be positive because QF = aii if the ith element of x is 1 and the rest are zeros. For example, for A 3 by 3, x⊤ = (0 1 0) yields QF = a22. A weaker requirement is that A be positive semi-definite. This means QF ≥ 0 for all x vectors, whether they are all zeros or not. Here the same logic as above implies that the main diagonal of a positive semi-definite matrix must be positive numbers or zeros. From the definitions, every positive definite matrix must be positive semi-definite. To obtain the definitions of negative definite and negative semi-definite change greater than signs to less than signs.

1

There are n by n matrices that are neither positive nor negative semi-definite; for example, any matrix whose main diagonal has positive and negative numbers. These matrices are indefinite. Are there necessary and sufficient conditions that determine whether a matrix is some form of “definite” or not? The answer is yes. Let A be a symmetric n by n matrix. The leading principal minors of A are the determinants of matrices obtained by deleting the last k rows and columns of A. If k = 0 the leading principal minor is det(A). If A is 3 by 3 the leading principal minors are det(A) for 2 for k = 1 and a k = 0, a11a22 − a12 11 for k = 2. One theorem is that A is positive definite if and only if all the leading principal minors of A are positive. If the leading principal minors alternate in sign with the first negative, that is, a11 < 0, a11 a22 − a122 > 0, and so on, then A is negative definite. The theorem for semi-definite matrices uses permutations. An ij permutation of A first interchanges the ith and jth rows of A and then interchanges ith and jth columns of A. Recall that if B = A except in B two rows or columns of A have been interchanged, then det(B) = − det(A). Thus a permutation does not change the determinant of A because interchanging two rows in A will change the sign of the determinant but then interchanging the corresponding columns will change the sign again, back to its original value. To see how permutations work let   a11 a12 a13 A =  a21 a22 a23  . a31 a32 a33 Interchanging the first and  a21  a11 a31

second rows and then the first and   a22 a23 a22 a21   a12 a13 a12 a11 and then a32 a33 a32 a31

second columns we obtain  a23 a13  . a33

For i = 1 and j = 3 we end up with 

 a33 a32 a31  a23 a22 a21  . a13 a12 a11

For i = 2 and j = 3 we end up with 

 a11 a13 a12  a31 a33 a32  . a21 a23 a22

Looking at matrix A and its permutations there are 7 distinct leading principal minors: three arising from one by one matrices — a11, a22 , a33 ; three arising from two by two matrices, 2 , a22 a33 −a223; and then det(A). (Note that we are using the symmetry a11a22 − a212, a11 a33 −a13 of A here.) A matrix is positive semi-definite if and only if the leading principal minors of this matrix and all its permutations are greater than or equal to zero. If the leading principal minors alternate in sign with the determinants of all the one by one matrices ≤ 0, the

2

determinants of all the two by two matrices ≥ 0, and so on, then the matrix is negative semi-definite. Examples 1. State whether the following matrices are positive or negative, definite or semi-definite, or indefinite, and explain your answers carefully.       1 0 1 1 0 1 1 1 0 (a)  1 4 2  (b)  0 1 1  (c)  0 1 1  2 0 2 1 1 2 0 2 3 ANSWER (a) Expanding down the first column the determinant of this three by three matrix is 1(8) + 1(−3)) = 5, so the leading principal minors are 1, 3, 5. This means the matrix is positive definite. (b) Expanding down the first column the determinant of this three by three matrix is 1(1) + 1(−1) = 0. So the matrix cannot be positive definite but with positive elements on its main diagonal it could be positive semi-definite. We must check aii ajj − aij2 for all i not equal to j. Checking we see that 2 = 1 a11a22 − a12 2 a11a33 − a13 = 1 2 a22a33 − a23 = 1,

so yes it is positive semi-definite. (c) This matrix is not symmetric but we know its “definiteness” is the same as that of the symmetric matrix B formed by averaging A’s off-diagonal elements. Thus   1 0 3/2 B= 0 1 1/2  3/2 1/2 2 Expanding down the first column the determinant of B is (1)(7/4)+(3/2)(−3/2) = −1/2 < 0. Given this and positive elements on the main diagonal the matrix must be indefinite. 2. State whether the matrices associated with the following quadratic forms are positive or negative, definite or semi-definite, or indefinite, and explain your answers carefully. (a) (b) (c)

6x12 + 25x22 + 9x32 − 60x2 x3 + 40x1 x3 − 6x1 x2 9x22 + 9x23 + 10x2 x3 + 6x1 x2 3x12 + 2x22 + x23 + 4x1 x2 + 4x2 x3 3

ANSWER (a) The matrix associated with this QF is   6 −3 20 A =  −3 25 −30  20 −30 9 Expanding along the first row, the determinant of A is 6(225 − 900) − 3(27 − 600) + 20(90 − 500) < 0; with positive elements on the main diagonal the matrix must be indefinite. (b) The matrix associated with this QF is   0 3 0 A= 3 9 5  0 5 9 Expanding along the first row, the determinant of A is 3(−27) < 0; with two positive elements on the main diagonal the matrix must be indefinite. (c) The matrix associated with this QF is   3 2 0 A= 2 2 2  0 2 1 Expanding along the first row, the determinant of A is 3(−2) + 2(−2) < 0; with positive elements on the main diagonal the matrix must be indefinite.

4...