Online PassiveAggressive Algorithms University of Chicago在线被动攻击算法芝加哥大学.pptVIP

Online Passive-Aggressive Algorithms Shai Shalev-Shwartz joint work with Koby Crammer, Ofer Dekel Yoram Singer The Hebrew University Jerusalem, Israel Three Decision Problems Online Setting Receive instance n/a Predict target value Receive true target ; suffer loss Update hypothesis A Unified View Define discrepancy for : Unified Hinge-Loss: Notion of Realizability: A Unified View (Cont.) Online Convex Programming: Let be a sequence of convex functions: Let be an insensitivity parameter. For Guess a vector Get the current convex function Suffer loss Goal: minimize the cumulative loss The Passive-Aggressive Algorithm Each example defines a set of consistent hypotheses: The new vector is set to be the projection of onto Passive-Aggressive An Analytic Solution Loss Bounds Theorem: - a sequence of examples. Assumption: Then if the online algorithm is run with , the following bound holds for any where for classification and regression and for uniclass. Loss bounds (cont.) For the case of classification we have one degree of freedom since if then for any Therefore, we can set and get the following bounds: Loss bounds (Cont). Classification Uniclass Proof Sketch Define: Upper bound: Lower bound: Proof Sketch (Cont.) Combining upper and lower bounds The Unrealizable Case Main idea: downsize step size by Loss Bound Theorem: - sequence of examples. bound for any and for any  Implications for Batch Learning Batch Setting: Input: A training set , sampled i.i.d according to an unknown distribution D