SSP19 Solution 5 - Lösung von Tutorial 5 PDF

Preview

Summary

Lösung von Tutorial 5...

Description

5. Linear Estimation

5.1

LMMSE Estimator, MSE and SNR

a) The estimate xˆ LMMSE = T TLMMSE y + mLMMSE with T TLMMSE ∈ RNT ×NR and mLMMSE ∈ RNT is determined as the solution of the unconstrained convex optimization problem given as    n h io T  2 T T −T −m −T −m argmin E k − ˆLMMSE k 2 = argmin tr E T,m T,m ioo n n h T − T T − mT − T T T + T T T T + T T m − m T + m T T + mmT =argmin tr E T,m n n oo =argmin tr R − R T − µ mT − T T R + TT R T + T Tµ mT − mµT + mµT T + mmT T,m n n oo =argmin tr −2T T R − 2mµT + 2T T µ mT + T T R T + mmT . T,m

Setting the derivatives of the objective to zero gives ∂ n tr −2T T R ∂T T

o − 2mµT + 2TT µ mT + T T R T + mmT = −2RT + 2mµT + 2T T R = −2R

!

+ 2TT R + 2mµT = 0

and ∂ n tr −2T T R ∂m

o ! − 2mµT + 2TT µ mT + TT R T + mmT = −2µ + 2m + 2TT µ = 0.

For m, we obtain m∗ = µ − TT µ which results in −R

+ TT R + (µ − T T µ )µT = −(R = −C

− µ µT ) + TT (R − µ µT) !

+ TTC = 0

Thus, we obain T ∗,T = C C−1 As the optimization problem is convex, the stationary point is a minimizer and T = T ∗,T = C C−1 TLMMSE

mLMMSE = m∗ = µ − T T µ . 62

5. Linear Estimation

63

This directly leads to the LMMSE estimate ˆ LMMSE for

based on

given as

ˆLMMSE = TTLMMSE + mLMMSE = µ + C C−1( − µ ). First, note that in our example as well as have zero mean. Therefore, From the system model, we have that i i h h T C =E = E (H + ) T = C HT

has zero mean as well.

and

C =E which results in

h

T

i

i h = E (H + ) (H + )T = HC HT + C ,

−1  T TLMMSE . = C HT HC HT + C

The random vectors and are jointly Gaussian due to the linear channel model = H + . In h iT T particular, the random vector = T is a Gaussian random vector, as it can be constructed using the affine transformation " # " #" # H I = = I 0 iT h T of the Gaussian, due to the independence of and , random vector T . Thus, the conditional mean estimator, i.e., the estimator which minimizes the MSE without the constraint that it has to be linear, turns out to be a linear estimator. Therefore, the LMMSE estimator is an MMSE estimator as well. All ingredients for the MMSE estimator can be found in the first and second order moment of the Gaussian random vector . Using # "" ## " # " µ Hµ + µ µ =E = = µ µ and "

C C = C

C C

#

"

HC HT + C = C HT

# HC . C

This results in the MMSE estimator, here for the general case, given as xˆ MMSE = E [ | ] = xˆ LMMSE   = µ + C C−1 y − µ . b) For NT = 1, it follows that C ⇒ σ2 and H ⇒ h ∈ RNR×1 . The filtered receive signal reads as tT =

tTh |{z}

+ |{z} tT .

filtered useful signal

filtered noise

5. Linear Estimation

64

A filter which maximizes the SNR in the presence of additive noise is called a matched filter (MF). The MF is determined by    2     2    2   T   2 T T     t h E       σ t h        t h      t MF = argmax    . = argmax = argmax     2         T T 2 N ×1 N ×1 t t t t N ×1 σ       E tT t∈R R t∈R R t∈R R     ( t T h) 2 Note that the objective function f (t) = t T t is scaling-invariant, i.e., we have that f (t) = f (ct ) for every c ∈ R. Therefore, we determine t MF by w.l.o.g. fixing the norm to ktk22 = 1. This results in the optimization problem  2  t MF = argmax (5.1) tTh t ∈RNR ×1 ,k t k22=1

which has a maximizer that is obviously colinear with h. Thus, the solution (5.1), i.e., the MF with unit norm is given as t MF =

h khk2

Equivalently, as the solution is arbitrary with respect to scaling, we can write t MF = ch ∝ h as every t proportional to h achieves the same SNR given as   2  2 2 T σ2 tMF c2 σ2 hT h σ2 chT h h σ2 hT h = . SNRMF = 2 T = 2 2 T = 2 T σ2 σ t MF t MF cσ h h σ ch ch c) For NT = 1, it follows that TLMMSE ⇒ t LMMSE ∈ RNR×1 . Thus, the estimator is given as  −1 T t LMMSE = c C−1 = σ2 hT hσ2 hT + σ2 I as c = σ2 hT and C = hσ2 hT + σ2 I for the case that NT = 1. Using the Sherman–Morrison T formula with A = σ2 I, B = h, and c = σ2 h, the filter tLMMSE can be expressed as −1  T tLMMSE = σ2 hT σ2 hhT + σ2 I  −1 = σ2 hT bcT + A 2

=σ h

T

A

−1

−

A−1 bcT A−1

!

1 + cT A−1 b ! σ−2 Ihσ2 hTσ−2 I −2 2 T =σ h σ I− 1 + σ2 hTσ−2Ih ! σ−2 hσ2 hT σ−2 2 T −2 =σ h σ I− 1 + σ2 hT σ−2h

5. Linear Estimation

65 2 σ2 σ−2hT h Tσ − h σ2 σ−2 + σ−2 hT h σ2 2 σ−2 Tσ = h σ−2 + σ−2 hT h σ2

= hT

=

hT σ2 σ2

∝ hT .

T

+h h

Now, we observe that in this special case the LMMSE filter is proportional to the MF, i.e., t LMMSE ∝ t MF ∝ h, and thus achieves the same SNR as the MF, which is the optimal SNR. d) We have E [ˆLMMSE] = 0 due to E [ ] = 0 as E [ ] = 0 and E [ ] = 0. Thus, the mean square of the error = − ˆLMMSE of the LMMSE estimates is given as i h σ2 = E ( − ˆLMMSE) 2  2  T − t LMMSE =E i h T T t LMMSE + t LMMSE = E 2 − 2t TLMMSE T c = σ2 − 2tLMMSE −1 2 T = σ − 2c C c

+ t TLMMSEC t LMMSE + cT C−1 c

T = σ2 − tLMMSE c T hσ2 = σ2 − tLMMSE   T = σ2 1 − t LMMSE h .

Using the result for the LMMSE estimator t TLMMSE derived in sub-problem c) results in   T σ2 = σ2 1 − tLMMSE h   T   h h  = σ2 1 −   σ2  T h h + σ2 σ2 σ2

2

=σ =

2

hT h + σσ2 σ2

1+

σ2 hT h σ2

=

σ2 . 1 + SNRMF

5. Linear Estimation

5.2 a)

66

”Linear” Models and LMMSE Calculating the mean of E

h

"" ## ′

T

′,T

=E

""

iT

leads to

TT +

##

"

µ = T T µ +µ

#

" # µ = =0 µ !

which results in the mean of the noise vector given as µ = µ − TTµ = 0 as µ = 0 and µ = 0. Calculating the covariance matrix of

h

T

′,T

iT

leads to

"  " # " #T  #T  #"         E  ′ T  ′  = E  TT + T +   T  T   TT +    = E        T T T T T T + T + T + # " # " C T ! C CT C = T = C C T C TTC T + C which results in the ”channel” matrix TT = C C−1 and the ”noise” covariance matrix C = C − C C−1 C . Equivalently for the second ”linear” model, we can determine the mean of the noise µ = µ − STµ = 0, the ”channel” matrix ST = C C−1, and the ”noise” covariance matrix C = C − C C−1 C .

T b) Taking into account that all random variables are zero mean, the LMMSE estimator TLMMSE for an estimation of based on is given as

T TLMMSE = C C−1

5. Linear Estimation

67

and the LMMSE estimate ˆLMMSE for

based on

ˆLMMSE =

TTLMMSE

given as = C C−1 .

T The LMMSE estimator SLMMSE for an estimation of

STLMMSE and the LMMSE estimate ˆLMMSE for

based on

=C C

based on

is given as

−1

given as

ˆLMMSE = STLMMSE = C C−1 . T We observe that TLMMSE = T T and STLMMSE = ST .

c) As E [ ˆLMMSE ] = 0 and E [ ] = 0, the covariance matrix of the LMMSE estimates ˆLMMSE is given as i h T = T TLMMSEC T LMMSE = C C−1C C ˆLMMSE = E ˆLMMSE ˆ LMMSE

and the covariance matrix of the error − ˆLMMSE of the LMMSE estimates is given as   T  h i T C = E ( − ˆ LMMSE ) ( − ˆLMMSE )T = E − TTLMMSE − TLMMSE =E

h

T

− T TLMMSE

= C − T TLMMSEC

T

−

T + T TLMMSE TLMMSE

T

TLMMSE

i

− C TLMMSE + T TLMMSEC TLMMSE

= C − C C−1 C which is exactly the noise covariance matrix C as determined in sub-problem a). Note that C =C

= C − C ˆ LMMSE .

Repeating above steps for the second estimator gives C ˆLMMSE = C C−1C and C =C

−1 = C − C ˆLMMSE = C − C C C .

d) The optimal estimator with respect to the MSE is given by the conditional mean estimator. iT h If = T T is a Gaussian random vector, the conditional mean estimators are given by the means of the posterior PDFs f | and f | which are both Gaussian as well, cf. tutorials 1 and 4. The conditional mean estimators are, cf. problem sheet on Bayesian estimators, given as −1 yˆ CM = E [ | ] = C C x = TTLMMSE x

and T y xˆ CM = E [ | ] = C C−1 y = SLMMSE

Thus, the LMMSE estimators actually are optimal estimators with respect to the MSE if the joint distribution of and is Gaussian.

5. Linear Estimation

5.3

68

Least Squares MIMO Channel Estimation

N a) The receive vectors {yi }i=1 ∈ RNR are stacked into a matrix Y ∈ RN×NR by

  T  ——y 1 ——   .. Y =   .   T ——yN ——

N ∈ RNT are stacked into a matrix X ∈ RN×NT by and the transmit vectors {xi }i=1

   ——x1T——   ..  . X =  .   T ——x N ——

This leads to the linear model

ˆ LS = XT LS ∈ RN×NR Y in order to determine the linear least squares estimator T TLS : RNT → RNR , x 7→ yˆ LS = T TLS x.

b)

Applying the orthogonality principle column-wise, leads to the condition that we have Y − Yˆ LS = Y − XT LS ⊥ span(X).

which has to be interpreted column-wise. We have Y − Yˆ LS ∈ null(X T)

and Yˆ LS ∈ span(X),

which is meant column-wise as well. c)

Using the orthogonality condition from the previous sub-problem column-wise, it follows that Y − Yˆ LS = Y − XT LS ∈ null(X T)

which is equivalent to XT (Y − XTLS ) = 0NT ×NR , i.e., we require that every column of the matrix Y − XT LS is element of null(XT ). Above condition results in the well-known normal equation XTY = XT XT LS. The matrix X T X ∈ RNT ×NT is invertible if and only if rank(X) = NT , i.e., if and only if X ∈ RN×NT is a tall matrix as N ≥ NT and has full column rank NT . Then, the least squares estimator is given as −1  T LS = XT X XT Y

5. Linear Estimation

69

and the least squares estimates are given as  −1 Yˆ LS = X X T X XT Y. d)

Alternatively, the least squares estimator is given as the solution of   Y − ˆ 2 Y LS F T LS = argmin T∈RNT ×NR o n = argmin kY − XTk2F T∈RNT ×NR oo n n = argmin tr (Y − XT) (Y − XT)T T∈RNT ×NR oo n n = argmin tr YY T − Y(XT)T − XTY T + XT( XT)T T∈RNT ×NR n n oo = argmin tr −YTT XT − XTY T + XTT T XT . T∈RNT ×NR

n o n o n o n o As tr AT B = tr B AT = tr BT A = tr ABT , it follows that

n n oo n n oo T LS = argmin tr −YTT XT − XTY T + XTT T XT = argmin tr −2T T XT Y + T T XT XT T∈RNT ×NR

T∈RNT ×NR

which can be, for example, solved by using the first and second derivative with respect to T T . The first derivative is given as   T ∂  ! −2T T XT Y + T T XT XT = −2 XT Y + 2T T X T X = 0 T ∂T which results in the unique stationary point T ∗,T given as T  −1  T ∗,T = XT Y XT X . The inverse 

XT X

−1

exists, if X T X is positive definite which is the case if and only if rank(X) = NT , i.e., if and only if X ∈ RN×NT is a tall matrix as N ≥ NT and has full column rank NT . Then, the stationary point T ∗,T is a minimizer and we have −1    TLS = T ∗,T = XT X XT Y.

5. Linear Estimation

5.4

70

Least Squares Estimation–A Different Perspective

a) Note that h

 i  . . . xN  

h

 i  . . . yN  

X T X = x1 and equivalently Y T X = y1 Thus, we have

  X N  xi xTi = NRˆ xx  =  i=1

x1T .. . xTN

  X N  yi xiT = N Rˆ yx .  =  i=1

x1T .. . xNT

(5.2)

(5.3)

T ˆ yx (N R ˆ xx )−1 = Rˆ yx Rˆ −1 TLS = Y T X(X T X)−1 = N R xx

(5.4)

−1 ˆ yx Rˆ xx yˆ LS = R x.

(5.5)

and

b) If E[x] = 0 and E[y] = 0, the sample correlation matrices are estimates for the covariance matrices for a known mean of zero and we have −1 T ˆ xx = Cˆ yx C TLS

and

−1 yˆ LS = Cˆ yx Cˆ xx x.

(5.6)

Thus, if E[x] = 0 and E[y] = 0, the linear LS estimate can be interpreted as an approximation of the LMMSE estimate, where the true covariance matrices are replaced by estimates for the covariance matrices. c) If E[x] = 0, E[y] = 0 and N → ∞, the sample correlation matrices converge (in probability) to the true covariance matrices. By this means, the LS estimator converges to the LMMSE estimator. d) We have

and

i h h i h i ˆ yx µˆ y Y T X ′ = Y T X 1 = Y T X YT 1 = N R

(5.7)

" T # # # " i ˆ xx µˆ x X X XT1 XT h R . X X = T X 1 = T =N µˆ xT 1 1 X 1T 1 1

(5.8)

′,T

′

"

Using the provided hints, we obtain = Y T X′ (X′,T X′ )−1 T ′,T LS #!−1 " i ˆ xx µˆ x R ˆ = N Ryx µˆ y N T µˆ x 1 h

5. Linear Estimation

71 #! i "R ˆ xx µˆ x −1 µˆ Tx 1 # i " (R ˆ xx − µˆ x µˆxT)−1 µˆ ˆ xx − µˆ x µˆxT)−1 −( R x µˆ y −µˆ xT(Rˆ xx − µˆ x µˆ xT)−1 1 + µˆ Tx (Rˆ xx − µˆ x µˆ xT)−1 µˆ x   −1 i  Cˆ −1 ˆ xx µˆ x  −C xx  µˆ y  T −1  −1 −µˆ xCˆ xx 1 + µˆ xTCˆ xx µˆ x

h

= Rˆ yx µˆ y h = Rˆ yx h = Rˆ yx

(5.9)

as Cˆ xx = Rˆ xx − µˆ x µˆ xT. Now, it directly follows that ′,T TLS

  −1 i  Cˆ −1  ˆ µ ˆ C −  xx x  ˆ yx µˆ y  xx −1 = R ˆ xx 1 + µˆ xT C ˆ −1 µˆ  −µˆ xT C xx x h i  −1   T ˆ −1 ˆ ˆ = Ryx − µˆ y µˆ x Cxx − R yx + µˆ y µˆ Tx Cˆ xx µˆ x + µˆ y i h −1 ˆ yxCˆ −1 ˆ ˆ = C C µ ˆ + µ ˆ − C yx xx x xx y h

(5.10)

ˆ yx − µˆ y µˆ Tx and thus as Cˆ yx = R yLS = =

h

ˆ yxCˆ −1 C xx

−1 ˆ yx C ˆ xx C x

= µˆ y +

−1 −Cˆ yxCˆ xx µˆ x

ˆ −1 ˆx − Cˆ yx C xx µ  −1  ˆ yx Cˆ C ˆx xx x − µ

+ µˆ y

i " x# 1

+ µˆ y (5.11)

Thus, the affine LS estimate can be interpreted as an approximation of the LMMSE estimate, where the true means and true covariance matrices are replaced by the respective sample means and sample covariance matrices. e) If N → ∞, the sample means and sample covariances converge (in probability) to the true means and covariances. Thus, by this means, the LS estimator converges to the LMMSE estimator....