Constrained estimation and the theorem of Kuhn-Tucker.pdfVIP

CONSTRAINED ESTIMATION AND THE THEOREM OF KUHN-TUCKER ORI DAVIDOV Received 11 July 2004; Accepted 11 January 2005 We explore several important, and well-known, statistical models in which the estimation procedure leads naturally to a constrained optimization problem which is readily solved using the theorem of Kuhn-Tucker. Copyright © 2006 Ori Davidov. This is an open access article distributed under the Cre- ative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 1. Introduction and motivation There are many statistical problems in which the parameter of interest is restricted to a subset of the parameter space. The constraint(s) may reflect prior knowledge about the value of the parameter, or, may be a device used to improve the statistical prop- erties of the estimator. Estimation and inferential procedures for such models may be derived using the theorem of Kuhn-Tucker (KT). The theorem of KT is a theorem in nonlinear programming which extends the method of Lagrange multipliers to inequality constraints. KT theory characterizes the solution(s) to general constrained optimization problems. Often, this characterization yields an algorithmic solution. In general, though, this is not the case and the theorem of KT is used together with other tools or algo- rithms. For example, if the constraints are linear or convex, then the tools of convex optimization (Boyd and Vandenberghe [2]) may be used; of these linear and quadratic programming are best known. More generally, interior point methods, a class of itera- tive methods in which all iterations are guaranteed to stay within the feasible set, may be used. Within this class, Lange [12] describes the adaptive barrier method with statistical applications. Geyer and Thompson [6] develop a Monte-Carlo method for constrained estimation based on a simulation of the likelihood function. Robert