Information Theoretic Bounds on Sparsity Recovery in the High Dimensional and Noisty Setting英文清晰版.pdfVIP

Appeared at International Symposium on Information Theory, Nice, France. June 2007. Information-theoretic bounds on sparsity recovery in the high-dimensional and noisy setting Martin Wainwright Department of Statistics, and Department of Electrical Engineering, UC Berkeley, Berkeley, CA 94720 wainwrig@{eecs,stat} .berkeley.edu achieve the information-theoretic limits. On the other hand, an Abstract— The problem of recovering the sparsity pattern information-theoretic analysis can reveal where there currently of a fixed but unknown vector β ∗ ∈ Rp based on a set of exists a gap between the performance of computationally n noisy observations arises in a variety of settings, including tractable methods, and the fundamental limits. Indeed, the subset selection in regression, graphical model selection, signal denoising, compressive sensing, and constructive approximation. information-theoretic analysis of this paper makes contribu- Of interest are conditions on the model dimension p, the sparsity tions of both types. index s (number of non-zero entries in β ∗), and the number With this motivation in mind, the focus of this paper is of observations n that are necessary and/or sufficient to ensure on the information-theoretic limitations of sparsity recovery. asymptotically perfect recovery of the sparsity patter