ReviewofProbabilityandStatistics-UniversityofNotreDame.pptVIP

The Simple Regression Model y = b0 + b1x + u Some Terminology In the simple linear regression model, where y = b0 + b1x + u, we typically refer to y as the Dependent Variable, or Left-Hand Side Variable, or Explained Variable, or Regressand Some Terminology, cont. In the simple linear regression of y on x, we typically refer to x as the Independent Variable, or Right-Hand Side Variable, or Explanatory Variable, or Regressor, or Covariate, or Control Variables A Simple Assumption The average value of u, the error term, in the population is 0. That is, E(u) = 0 This is not a restrictive assumption, since we can always use b0 to normalize E(u) to 0 Zero Conditional Mean We need to make a crucial assumption about how u and x are related We want it to be the case that knowing something about x does not give us any information about u, so that they are completely unrelated. That is, that E(u|x) = E(u) = 0, which implies E(y|x) = b0 + b1x . . x1 x2 E(y|x) as a linear function of x, where for any x the distribution of y is centered about E(y|x) E(y|x) = b0 + b1x y f(y) Ordinary Least Squares Basic idea of regression is to estimate the population parameters from a sample Let {(xi,yi): i=1, ?n} denote a random sample of size n from the population For each observation in this sample, it will be the case that yi = b0 + b1xi + ui . . . . y4 y1 y2 y3 x1 x2 x3 x4 } } { { u1 u2 u3 u4 x y Population regression line, sample data points and the associated error terms E(y|x) = b0 + b1x Deriving OLS Estimates To derive the OLS estimates we need to realize that our main assumption of E(u|x) = E(u) = 0 also implies that Cov(x,u) = E(xu) = 0 Why? Remember from basic probability that Cov(X,Y) = E(XY) ?E(X)E(Y) Deriving OLS continued We can write our 2 restrictions just in terms of x, y, b0 and b1 , since u = y ?b0 ?b1x E(y ?b0 ?b1x) = 0 E[x(y ?b0 ?b1x)] = 0 These are called moment restrictions Deriving OLS