MULTIPLE REGRESSION ANALYSIS: THE PROBLEM OF ESTIMATION PDF

Preview

Summary

MULTIPLE REGRESSION ANALYSIS: THE PROBLEM OF ESTIMATION Fungsi regresi populasi dengan 3 variabel dapat ditulis sbb: (7.1.1) Yi = β 1+ β 2 X 2i + β 3 X 3i + ui asumsi: nilai rata-rata dari variabel ui adalah nol E (ui | X 2i , X 3i ) = 0 for each i (7.1.2) tidak ada autokorelasi antar variabel gangg...

Description

MULTIPLE REGRESSION ANALYSIS: THE PROBLEM OF ESTIMATION

Fungsi regresi populasi dengan 3 variabel dapat ditulis sbb: (7.1.1)

Yi = β 1+ β 2 X 2i + β 3 X 3i + ui

asumsi: nilai rata-rata dari variabel ui adalah nol E (ui | X 2i , X 3i ) = 0 for each i

(7.1.2)

tidak ada autokorelasi antar variabel gangguan cov(ui , u j ) = 0

i≠ j

(7.1.3)

varians konstan (Homoscedasticity) var(ui ) = σ 2

(7.1.4)

covariance antara ui dan setiap variabel X adl nol cov(ui , X 2i ) = cov(ui , X 3i ) = 0

(7.1.5)

tidak ada bias spesifikasi Model telah terspesifikasi dengan benar

(7.1.6)

tidak kolinearitas diantara variabel X, atau tidak ada hubungan linear yang pasti antara X2

(7.1.7)

dan X3 (tidak ada multikolinearitas) Jumlah observasi (n) harus lebih besar daripada jumlah parameter Harus terdapat variasi pada nilai variabel X Linear dalam parameter

Interpretasi Persamaan Regresi Berganda

E (ui | X 2i , X 3i ) = β1 + β 2 X 2i + β 3 X 3i Persamaan tsb menggambarkan rata – rata kondisional atau nilai ekspektasi kondisional dari Y terhadap nilai tertentu dari X2 dan X3

ARTI KOEFISIEN REGRESI PARSIAL

β2 mengukur perubahan nilai rata -rata Y, E(Y), untuk setiap perubahan X2, dengan asumsi X3 konstan. β3 mengukur perubahan nilai rata-rata Y untuk setiap perubahan X3, dengan asumsi nilai X2 konstan.

OLS Estimators βˆ2 = βˆ3

(

yi x2i )(

(

x32i ) − (

x2i x3i )

)− ( x x ) ( y x )( x ) − ( y x )( x = ( x )( x ) − ( x x ) x

2 2i

)(

yi x3i )(

x

2 i 3i

2 2i

i 3i

2 2i

(7.4.7)

2

2 3i

i 2i

x

2 i 3i

)

(7.4.8)

2

2 3i

2 i 3i

Variances and Standard Errors of OLS Estimators 1 var(βˆ1 ) = + n

X 22

se( βˆ1 ) = + var(βˆ1 )

x22i − 2 X (

x32i + X 32

(

x

2 2i

)(

2 3i

x

)− (

yi x3i )( x2i x3i )

2

x2i x3i )

.σ 2 (7.4.9)

(7.4.10)

var(βˆ2 ) =

x32i

(

x22i )(

x32i ) − (

x2i x3i )

or, equivalently, var(βˆ2 ) =

2 σ 2

σ2 x22i (1 − r223 )

(7.4.11)

(7.4.12)

(7.4.13)

se( βˆ1 ) = + var(βˆ1 )

var(βˆ3 ) = or, equivalently,

x22i

(

x22i )(

var(βˆ3 ) =

2 σ 2

x32i ) − (

x2i x3i )

(7.4.14)

)

(7.4.15)

σ2 x (1 − r 2 3i

2 23

se( βˆ3 ) = + var(βˆ1 )

cov(βˆ2 , βˆ3 ) =

(7.4.16)

− r2 3σ 2

(1 − r ) 2 23

2 u ˆ i σˆ 2 = n−3

x22i x32i

(7.4.17) (7.4.18)

THE MULTIPLE COEFFICIENT OF DETERMINATION R2 AND THE MULTIPLE COEFFICIENT OF CORRELATION R ESS R = TSS βˆ2 yi x2i +βˆ3 = yi2 2

(7.5.5)

yi x3i

R2 AND THE ADJUSTED R2 ESS R = TSS RSS = 1− TSS uˆi2 = 1− yi2 2

(7.8.1)

Untuk membandingkan dua jenis R2, kita harus memperhatikan jumlah variabel X yang terdapat dalam model. Hal ini dgn mudah dapat diatasi jika kita mempertimbangkan alternative coefficient of determination sbb: 2 u ˆ i (n − k ) 2 (7.8.2) R = 1− 2 yi (n − 1) dimana k = jumlah parameter dalam model termasuk intercept.

2 2 Kita dapat melihat R and R memiliki keterkaitan, yakni dgn mensubtitusi pers (7.8.1) ke dalam pers (7.8.2), diperoleh:

R

2

(

= 1− 1− R

2

( n − 1) ) (n − k )

(7.8.4)

R 2 Didefinisikan sebagai R2 yang disesuaikan dengan derajat kebebasannya (n – k). 1. jika k >1, R 2 < R2, yang mengimplikasikan bahwa : ketika jumlah variabel X meningkat maka R 2 peningkatan akan lebih kecil dibanding peningkatan R2 2. R 2 dapat bernilai negatif walau R2 tidak negatif

Contoh:

Homework:

Kerjakan soal 7.16. dikumpulkan minggu depan!

MULTIPLE REGRESSION ANALYSIS: THE PROBLEM OF INFERENCE

Upon replacing σ2 by its unbiased estimator… in the computation of the standard errors, each of the following variables follows the t distribution with n – 3 df. βˆ1 − β1 t= (8.1.1) ˆ ( ) se β1

βˆ2 − β 2 t= se(βˆ2 )

(8.1.2)

βˆ1 − β1 t= se(βˆ1 )

(8.1.3)

follows the t distribution with n – 3 df.

HYPOTHESIS TESTING ABOUT INDIVIDUAL REGRESSION COEFFICIENTS H0:β2 = 0

and

H1: β2 ≠ 0

The null hypothesis states that, with X3 (female literacy rate) held constant, X2 (PGNP) has no (linear) influence on Y (child mortality). For our illustrative example, using (8.1.2) and noting that β2 = 0 under the null hypothesis, we obtain − 0.0056 t= = −2.8187 0.0020

(8.4.1)

TESTING THE OVERALL SIGNIFICANCE OF THE SAMPLE REGRESSION Consider the following hypothesis: H0: β2 = β3 = 0

(8.5.1)

This null hypothesis is a joint hypothesis that β2 and β3 are jointly or simultaneously equal to zero. A test of such a hypothesis is called a test of the overall significance of the observed or estimated regression line, that is, whether Y is linearly related to both X2 and X3.

Although the statements Pr βˆ2 − tα 2 se(βˆ2 )≤ β 2 ≤ βˆ2 + tα 2 se(βˆ2 ) = 1 − α

[ Pr [βˆ

3

− tα 2 se(βˆ3 )≤ β 3 ≤ βˆ3 + tα 2

] se(βˆ )] = 1 − α 3

are individually include β2 and β3 is (1 – α)2, because the intervals my not be independent when the same data are used to derive them.

[βˆ

2

± tα 2 se(βˆ2 ), βˆ3 ± tα 2 se(βˆ3 )

]

simultaneously include β2 and β3 is (1 – α)2, because the intervals may not be independent when the same data are used to derive them.

The Analysis of Variance Approach to Testing he Overall Significance of an Observed Multiple Regression: The F Test TABEL 8.1

ANOVA TABLE FOR THE THREE-VARIABLE REGRESSION

Source of variation Due to regression (ESS)

SS

βˆ2

yi x2i +βˆ3

Due to residuals (RSS)

uˆ

Total

df 2

yi x3i

βˆ2

n–3

2 i

y

MSS yi x2i +βˆ3 2

yi x3i

2 u ˆ i σˆ 2 = n−3

n–1

2 i

Recall the identity yi2 =βˆ2

yi x2i +βˆ3 TSS =

yi x3i +

ESS

uˆi2

+ RSS

(8.5.2)

TSS has, as usual, n – 1 df and RSS has n – 3 df for reasons βˆ2 already discussed. ESS has 2 df since it is a function of……… and βˆ3 .Therefore, following the ANOVA procedure discussed in Section 5.9, we can set up Table 8.1. Now it can be shown that, under the assumption of normal distribution for ui and the null hypothesis H0: β2 = β3 = 0, the variable

( βˆ F=

2

yi x2i +βˆ3 yi x3i ) 2 ESS df = 2 uˆi (n − 3) RSS df

is distributed as the F distribution with 2 and n – 3 df.

(8.5.3)

An Important Relationship between R2 and F The null hypothesis is H0: β2 = β3 = … = βk = 0

(8.5.9)

then it follows that

ESS (k − 1) F= RSS (n − k )

(8.5.7) = (8.5.10)

Let us manipulate (8.5.10) as follows:

n − k ESS k − 1 RSS n−k ESS = k − 1 TSS − ESS n − k ESS / TSS = k − 1 1 − ( ESS / TSS )

F =

n − k R2 = k −1 1 − R2 R 2 (k − 1) = (1 − R 2 ) (n − k )

(8.5.11)...