January 13, 2014, 4:30pm
James Renegar, Operations Research and Information Engineering, Cornell University.
Extending the Applicability of Efficient First-Order Methods for Convex Optimization
Abstract: The study of first-order methods has largely dominated research in continuous optimization for the last decade, yet still the range of problems for which efficient and easily-applicable first-order methods have been developed is surprisingly limited, even though much has been achieved in some areas with high profile, such as compressed sensing.
We present a simple transformation of any linear or semidefinite (or hyperbolic) program into an equivalent convex optimization problem whose only constraints are linear equations. The objective function is defined on the whole space, making virtually all subgradient methods be immediately applicable. We observe, moreover, that the objective function is naturally smoothed, thereby allowing most first-order methods to be applied.
We develop complexity bounds in the unsmoothed case for a particular subgradient method, and in the smoothed case for Nesterov’s original optimal first-order method for smooth functions. We achieve the desired bounds on the number of iterations.
Perhaps most surprising is that the transformation is simple and so is the basic theory, and yet the approach has been overlooked until now, a blind spot.
Bio: James Renegar has been a Professor in the School of Operations Research and Information Engineering at Cornell University since 1987, and has served as its Director (2004–2009). His research interests lie in continuous optimization, and more broadly, in the interplay between algorithms, geometry, and algebraic techniques. Professor Renegar has made fundamental contributions to the design and analysis of interior point methods, error analysis in convex optimization, and numeric and algebraic computational complexity. He was one of the five founding members of the Society for Foundations of Computational Mathematics, and has long been on its board of directors. He was the chair of the workshop committee at the last meeting of the society (2014). Professor Renegar was a Semi-Plenary speaker at the 17th International Symposium on Mathematical Programming (2000) and an invited speaker at the International Congress of Mathematicians (1990). In addition, he has published an influential book on the mathematics of interior point methods (SIAM 2001) and has received numerous teaching awards.