A simpler approach to obtaining an O(1/t) convergence rate for projected stochastic subgradient descent

View Researcher II's Other Codes

Disclaimer: The provided code links for this paper are external links. Science Nest has no responsibility for the accuracy, legality or content of these links. Also, by downloading this code(s), you agree to comply with the terms of use as set out by the author(s) of the code(s).

Authors Simon Lacoste-Julien, Mark W. Schmidt, Francis R. Bach
Journal/Conference Name ARXIV: LEARNING
Paper Category
Paper Abstract In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to existing techniques, with similar performance behavior.
Date of publication 2012
Code Programming Language MATLAB/C
Comment

Copyright Researcher II 2021