AI Seminar: Gradient-free bilevel programming with applications in machine learning

Zoom: https://oregonstate.zoom.us/j/96491555190?pwd=azJHSXZ0TFQwTFFJdkZCWFhnT…

Our talk is mainly about a scalable and easy way of addressing bilevel optimization problems when we only have access to noisy evaluations of the objective functions, and absolutely no access to their gradient or Hessian. The talk starts with some motivating applications of bilevel programming in machine learning. We will then briefly overview zeroth-order methods for general optimization problems, and specifically focus on a more recent technique called Gaussian smoothing, which allows estimating the first and second order derivatives of an objective function solely through random queries to the function. Finally, we propose a computational algorithm to address general bilevel programs, without the need to access the Jacobian or Hessian of the underlying objectives. We present theoretical guarantees on the performance of the algorithm, and ultimately present iteration complexity (sample complexity) results. To the best of our knowledge, the proposed algorithm is the first result in the literature about a fully zeroth-order scheme to address general bilevel programs.