Chapter1–LinearRegressionwith1Predictor.docVIP

Chapter 1 – Linear Regression with 1 Predictor Statistical Model where: is the (random) response for the ith case are parameters is a known constant, the value of the predictor variable for the ith case is a random error term, such that: The last point states that the random errors are independent (uncorrelated), with mean 0, and variance . This also implies that: Thus, represents the mean response when (assuming that is reasonable level of ), and is referred to as the Y-intercept. Also, represent the change in the mean response as increases by 1 unit, and is called the slope. Least Squares Estimation of Model Parameters In practice, the parameters and are unknown and must be estimated. One widely used criterion is to minimize the error sum of squares: This is done by calculus, by taking the partial derivatives of with respect to and and setting each equation to 0. The values of and that set these equations to 0 are the least squares estimates and are labelled and . First, take the partial derivates of with respect to and : Next, set these these 2 equations to 0, replacing and with and since these are the values that minimize the error sum of squares: These two equations are referred to as the normal equations (although, note that we have said nothing YET, about normally distributed data). Solving these two equations yields: where and are constants, and is a random variable with mean and variance given above: The fitted regression line, also known as the prediction equation is: The fitted values for the individual observations aye obtained by plugging in the corresponding level of the predictor variable () into the fitted equation. The residuals are the vertical distances between the observed values () and their fitted values (), and are denoted as . Properties of the fitted regression line The residuals sum to 0 The sum of the weighted (by ) residuals is 0 The sum of the weighted (by ) residuals is 0 The regression line