Dual Coordinate Descent Algorithms for Efficient Large Margin有效大幅度的双坐标下降算法.pptx

Dual Coordinate Descent Algorithms for EfficientLarge Margin Structured Prediction Ming-Wei Chang and Scott Wen-tau Yih Microsoft Research 1 Motivation Many NLP tasks are structured Parsing, Coreference, Chunking, SRL, Summarization, Machine translation, Entity Linking,… Inference is required Find the structure with the best score according to the model Goal: a better/faster linear structured learning algorithm Using Structural SVM What can be done for perceptron? 2 Two key parts of Structured Prediction Common training procedure (algorithm perspective) Perceptron: Inference and Update procedures are coupled Inference is expensive But we only use the result once in a fixed step 3 Observations 4 Observations Inference and Update procedures can be decoupled If we cache inference results/structures Advantage Better balance (e.g. more updating; less inference) Need to do this carefully… We still need inference at test time Need to control the algorithm such that it converges 5 Questions Can we guarantee the convergence of the algorithm? Can we control the cache such that it is not too large? Is the balanced approach better than the “coupled” one? 6 Contributions We propose a Dual Coordinate Descent (DCD) Algorithm For L2-Loss Structural SVM; Most people solve L1-Loss SSVM DCD decouples Inference and Update procedures Easy to implement; Enables “inference-less” learning Results Competitive to online learning algorithms; Guarantee to converge [Optimization] DCD algorithms are faster than cutting plane/ SGD Balance control makes the algorithm converges faster (in practice) Myth Structural SVM is slower than Perceptron 7 Outline Structured SVM Background Dual Formulations Dual Coordinate Descent Algorithm Hybrid-Style Algorithm Experiments Other possibilities 8 Structured Learning 9 The Perceptron Algorithm 10 Structural SVM Objective function Distance-Augmented Argmax 11 Dual formulation 12 Outline Structured SVM Background Dual Formu