16. Mean Square Estimation.ppt

* 16. Mean Square Estimation Given some information that is related to an unknown quantity of interest, the problem is to obtain a good estimate for the unknown in terms of the observed data. Suppose represent a sequence of random variables about whom one set of observations are available, and Y represents an unknown random variable. The problem is to obtain a good estimate for Y in terms of the observations Let represent such an estimate for Y. Note that can be a linear or a nonlinear function of the observation Clearly represents the error in the above estimate, and the square of (16-1) (16-2) PILLAI the error. Since is a random variable, represents the mean square error. One strategy to obtain a good estimator would be to minimize the mean square error by varying over all possible forms of and this procedure gives rise to the Minimization of the Mean Square Error (MMSE) criterion for estimation. Thus under MMSE criterion,the estimator is chosen such that the mean square error is at its minimum. Next we show that the conditional mean of Y given X is the best estimator in the above sense. Theorem1: Under MMSE criterion, the best estimator for the unknown Y in terms of is given by the conditional mean of Y gives X. Thus Proof : Let represent an estimate of Y in terms of Then the error and the mean square error is given by (16-3) (16-4) PILLAI Since we can rewrite (16-4) as where the inner expectation is with respect to Y, and the outer one is with respect to Thus To obtain the best estimator we need to minimize in (16-6) with respect to In (16-6), since and the variable appears only in the integrand term, minimization of