Schedule

  • Event
    Date
    Description
    Course Material
  • Lecture
    09/15/2021
    Wednesday
    Lecture 1 Review

    Suggested Readings:

  • Lecture
    09/17/2021
    Friday
    Lecture 2 Duality Theory

    Suggested Readings:

    5.1, 5.2 and 5.4 of CVOPT

  • Lecture
    09/22/2021
    Wednesday
    Lecture 3 Conjugate Function I

    Suggested Readings:

    3.3 of CVOPT

  • Lecture
    09/24/2021
    Friday
    Lecture 4 Conjugate Function II

    Suggested Readings:

    3.3 of CVOPT

  • Assignment
    09/24/2021
    Friday
    Homework 1 released!
  • Lecture
    09/26/2021
    Sunday
    Lecture 5 KKT Optimality Conditions I

    Suggested Readings:

    5.5 of CVOPT

  • Lecture
    09/29/2021
    Wednesday
    Lecture 6 KKT Optimality Conditions II

    Suggested Readings:

    5.5 of CVOPT

  • Lecture
    10/08/2021
    Friday
    Lecture 7 Linear Programming I

    Suggested Readings:

  • Lecture
    10/13/2021
    Wednesday
    Lecture 8 Linear Programming II

    Suggested Readings:

  • Due
    10/13/2021 16:00
    Wednesday
    HW1 Due.
  • Assignment
    10/14/2021
    Thursday
    Homework 2 released!
  • Lecture
    10/15/2021
    Friday
    Lecture 9 Interior Point Method for Nonlinear Programming

    Suggested Readings:

  • Lecture
    10/20/2021
    Wednesday
    Lecture 10 Stochastic Gradient Descent I

    Suggested Readings:

    Section 8.7.1, 8.7.2, and 8.7.3 of Liu et al.

  • Lecture
    10/22/2021
    Friday
    Lecture 11 Stochastic Gradient Descent II

    Suggested Readings:

    Section 8.7.1, 8.7.2, and 8.7.3 of Liu et al.

  • Lecture
    10/27/2021
    Wednesday
    Lecture 12 Federated Optimization

    Suggested Readings:

    Li, Xiang, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. “On the Convergence of FedAvg on Non-IID Data.” In International Conference on Learning Representations. 2019.

    Kairouz, Peter, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz et al. “Advances and open problems in federated learning.” arXiv preprint arXiv:1912.04977 (2019).

  • Assignment
    10/27/2021
    Wednesday
    Homework 3 released!
  • Assignment
    10/27/2021
    Wednesday
    Homework 4 released!
  • Due
    10/27/2021 16:00
    Wednesday
    HW2 Due.
  • Lecture
    10/29/2021
    Friday
    Lecture 13 Stochastic Variance Reduced Gradient and Block Coordinate Descent

    Suggested Readings:

    Johnson, Rie, and Tong Zhang. “Accelerating stochastic gradient descent using predictive variance reduction.” Advances in neural information processing systems 26 (2013): 315-323.

    Wright, Stephen J. “Coordinate descent algorithms.” Mathematical Programming 151, no. 1 (2015): 3-34.

  • Lecture
    11/03/2021
    Wednesday
    Lecture 14 Alternating Deirection Method of Multipliers

    Suggested Readings:

    Boyd, Stephen, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. “Distributed optimization and statistical learning via the alternating direction method of multipliers.” Foundations and Trends in Machine Learning 3, no. 1 (2010): 1-122.

  • Lecture
    11/05/2021
    Friday
    Lecture 15 Course Review
  • Due
    11/05/2021 16:00
    Friday
    HW3 Due.
  • Due
    11/10/2021 16:00
    Wednesday
    HW4 Due.