作业五文件-Homework5forAlgorithmsForBig.PDF

Homework 5 for “Algorithms For Big Data Analysis” Zaiwen Wen Beijing International Center for Mathematical Research Peking University May 4, 2017 1 Submission Requirement 1. Prepare a report including detailed answers to each question numerical results and their iterpretation 2. The programming language can be either matlab, Python or c/c++. 3. Pack all of your codes named as ”hw5-ID-name.zip” and send it to both me and TA: wendouble@ pkuopt@163.com hw5- - 4. 5. word word pdf 6. If you get significant help from others on one routine, write down the source of references at the beginning of this routine. 2 Variants of Stochastic Gradients Algorithms Consider problem (2.1) where and . 1 1. Write down and implement two of the following algorithms: Adadelta, AdagradDA, Adagrad, ProximalAda- grad, Ftrl, Momentum, adam, Momentum, CenteredRMSProp, nesterov, rmsprop, SAG, SAGA, SVRG References: chapter 8 in: / 2. You are encouraged to read the implementation in caffe, tensorflow as well as other packages. However, you should implement the codes by yourself. If you are going to test stochastic gradient methods in project 1, please choose different algorithms other than these in project 1. 3. Data sets: MNIST and Covertype. The set up is exactly the same as section 5 in the following paper, except that the -norm regularization term is replaced by -norm. Note