Least Squares Algebra∶ Partial Regression and Correlation.pptVIP

* * * * * * * * * * * * Applied Econometrics William Greene Department of Economics Stern School of Busines Applied Econometrics 4. Least Squares Algebra: Partial Regression and Correlation Frisch-Waugh (1933) Theorem Context: Model contains two sets of variables: X = [ [1,time] | [ other variables]] = [X1 X2] Regression model: y = X1?1 + X2?2 + ? (population) = X1b1 + X2b2 + e (sample) Problem: Algebraic expression for the second set of least squares coefficients, b2 Partitioned Solution Method of solution (Why did FW care? In 1933, matrix computation was not trivial!) Direct manipulation of normal equations produces b2 = (X2?X2)-1X2?(y - X1b1) What is this? Regression of (y - X1b1) on X2 Important result (perhaps not fundamental). Note the result if X2?X1 = 0. Useful in theory: Probably Likely in practice? Not at all. Partitioned Inverse Use of the partitioned inverse result produces a fundamental result: What is the southeast element in the inverse of the moment matrix? Partitioned Inverse The algebraic result is: [ ]-1(2,2) = {[X2’X2] - X2’X1(X1’X1)-1X1’X2}-1 = [X2’(I - X1(X1’X1)-1X1’)X2]-1 = [X2’M1X2]-1 Note the appearance of an “M” matrix. How do we interpret this result? Note the implication for the case in which X1 is a single variable. (Theorem, p. 28) Note the implication for the case in which X1 is the constant term. (p. 29) Frisch-Waugh Result Continuing the algebraic manipulation: b2 = [X2’M1X2]-1[X2’M1y]. This is Frisch and Waugh’s famous result - the “double residual regression.” How do we interpret this? A regression of residuals on residuals. “We get the same result whether we (1) detrend the other variables by using the residuals from a regression of them on a constant and a time trend and use the detrended data in the regression or (2) just include a constant and a time trend in the regre