Lectures

You can download the lectures here. We will try to upload lectures prior to their corresponding classes.

• 

  Lecture 1 Introduction
  tl;dr: introduction lecture
  [slides] [notes]
  

  Suggested Readings:

  • LaTeX Materials
  • Section 1.1 and 1.5 of Liu et al.
• 

  Lecture 2 Modeling in Optimization
  tl;dr: Least Squares, MLE, Transportation Problem, Management Decision Tree Analysis, DL, RL
  [notes]
  

  Suggested Readings:

  • Section 3.1.1, 3.2, 3.3 and 3.13 of Liu et al.
• 

  Lecture 3 Algorithm and Theory in Optimization
  tl;dr: Management Decision Tree Analysis, RL, Algorithm and Theory Examples
  [notes]
  

  Suggested Readings:

  • Section 1.5.7 and 2.2.1 of Liu et al.
• 

  Lecture 4 Quick Review of Linear Algebra I
  tl;dr: Row and Column Picture, Matrix Multiplication, Vector Space, Vector and Matrix Norm and SVD
  [notes]
  

  Suggested Readings:

  • No Suggested Reading
• 

  Lecture 5 Quick Review of Linear Algebra II
  tl;dr: Row and Column Picture, Matrix Multiplication, Vector Space, Vector and Matrix Norm and SVD
  [notes]
  

  Suggested Readings:

  • No Suggested Reading
• 

  Lecture 6 Unconstrained Optimization-Gradient Descent Method I
  tl;dr: Gradient Descent with Line Search
  [notes]
  

  Suggested Readings:

  • Section 6.1.1 of Liu et al.
• 

  Lecture 7 Unconstrained Optimization-Gradient Descent Method II
  tl;dr: Gradient Descent for Beta Smooth Function and Convex Set
  [notes]
  

  Suggested Readings:

  • Section 6.2, 2.4 and 2.5 and of Liu et al.
• 

  Lecture 8 Unconstrained Optimization-Gradient Descent Method III
  tl;dr: Convex Function and GD for Beta-smooth and Convex Function
  [notes]
  

  Suggested Readings:

  • Section 6.2, 2.4 and 2.5 and of Liu et al.
• 

  Lecture 9 Unconstrained Optimization-Gradient Descent Method IV
  tl;dr: GD for Beta-Smooth and Alpha-Strongly Convex Function
  [notes]
  

  Suggested Readings:

  • Section 6.2, 2.4 and 2.5 and of Liu et al.
• 

  Lecture 10 Unconstrained Optimization-Subgradient Descent
  tl;dr: subgradient, subdifferential, subgradient descent
  [notes]
  

  Suggested Readings:

  • Section 6.3 and 2.7 and of Liu et al.
• 

  Lecture 11 Unconstrained Optimization-Proximal Gradient Descent I
  tl;dr: Ridge Regression, LASSO, Proximal Operator
  [notes]
  

  Suggested Readings:

  • Section 5.3.2, 8.1.1 and 8.1.2 and of Liu et al.
• 

  Lecture 12 Unconstrained Optimization-Proximal Gradient Descent II
  tl;dr: Proximal Operator, Proximal Gradient Descent
  [notes]
  

  Suggested Readings:

  • Section 5.3.2, 8.1.1 and 8.1.2 and of Liu et al.
• 

  Lecture 13 Unconstrained Optimization-Proximal Gradient Descent III
  tl;dr: Proximal Operator, Proximal Gradient Descent
  [notes]
  

  Suggested Readings:

  • Section 5.3.2, 8.1.1 and 8.1.2 and of Liu et al.
• 

  Lecture 14 Unconstrained Optimization-Accelerated Gradient Descent
  tl;dr: Accelerated Gradient Descent
  [notes]
  

  Suggested Readings:

  • Section 8.2 of Liu et al.
• 

  Lecture 15 Unconstrained Optimization-Newton Method
  tl;dr: Newton Method
  [notes]
  

  Suggested Readings:

  • Section 6.4 and 6.5 of Liu et al.
• 

  Lecture 16 Course Review
  tl;dr: Review