使用块坐标下降方案进行高效词典学习.docVIP

使用块坐标下降方案进行高效词典学习 刘宝弟，章毓晋 清华大学电子工程系，北京　100084 摘要：稀疏编码通常被看成是在非正交过完备基下重排数据以使得数据结构精简的方法。它广 泛应用于信号处理、模式识别、机器学习、统计以及神经科学领域。不幸的是，目前缺乏快速 高效地稀疏编码和过完备基学习算法。本文提出了使用块坐标下降算法求解稀疏编码和过完备 基学习问题首先，根据变量间的独立性，将优化问题分解成一系列块的优化问题；然后，对于 每个块的优化问题，根据抛物线函数的凸性和单调性直接求出闭式解。实验结果显示提出的高 效地词典学习算法不仅能够加速学习的过程，并且在实际的图像分类和人脸识别应用中有着较 大性能的提升。 关键词：信号与信息处理；稀疏编码；块坐标下降；图像分类 中图分类号： TP391.41 Blockwise Coordinate Descent Schemes for E?ective Dictionary Learning Liu Bao-Di, Zhang Yu-Jin Department of Electronic Engineering, Tsinghua University, Beijing 100084 Abstract: Sparse coding, which is usually viewed as a method for rearranging the structure of the original data in order to make the energy compact over non-orthogonal and overcomplete dictionary, is widely used in signal processing, pattern recognition, machine learning, statistics, and neuroscience. Unfortunately, ?nding sparse codes and learning bases remain computationally di?cult up to now, and the performance of sparse coding is sensitive to the learned dictionary. In this paper, we propose a blockwise coordinate descent algorithm with guaranteed convergence to solve these two problems under a uni?ed scheme. The variables involved in the optimization problems are partitioned into several suitable blocks with convexity preserved, making it possible to perform an exact block coordinate descent. For each separable subproblem, based on the convexity and monotonic property of the parabolic function, a closed-form solution is obtained. Thus the algorithm is simple, e?cient and e?ective. Experimental results show that our algorithm not only signi?cantly accelerates the learning process, but also greatly helps improve the performance of real applications. 基金项目： Ministry of Education under Grants (SRFDP-20110002110057) 作者简介： Liu Bao-Di，male，Ph.D. student，major research direction：Computer vision, machine learning and pattern recognition. Correspondence author：Zhang Yu-Jin，male，professor，major research direction：image engineering. lbd08@, zhang-yj@ -1- Key wor