Lectures
You can download the lectures here. We will try to upload lectures prior to their corresponding classes.
-
Lecture 1 Introduction
tl;dr: introduction lecture
[slides] [notes]
Suggested Readings:
- LaTeX Materials
- Section 1.1 and 1.5 of Liu et al.
-
Lecture 2 Modeling in Optimization
tl;dr: Least Squares, MLE, Transportation Problem, Management Decision Tree Analysis, DL, RL
[notes]
Suggested Readings:
- Section 3.1.1, 3.2, 3.3 and 3.13 of Liu et al.
-
Lecture 3 Algorithm and Theory in Optimization
tl;dr: Management Decision Tree Analysis, RL, Algorithm and Theory Examples
[notes]
Suggested Readings:
- Section 1.5.7 and 2.2.1 of Liu et al.
-
Lecture 4 Quick Review of Linear Algebra I
tl;dr: Row and Column Picture, Matrix Multiplication, Vector Space, Vector and Matrix Norm and SVD
[notes]
Suggested Readings:
- No Suggested Reading
-
Lecture 5 Quick Review of Linear Algebra II
tl;dr: Row and Column Picture, Matrix Multiplication, Vector Space, Vector and Matrix Norm and SVD
[notes]
Suggested Readings:
- No Suggested Reading
-
Lecture 6 Unconstrained Optimization-Gradient Descent Method I
tl;dr: Gradient Descent with Line Search
[notes]
Suggested Readings:
- Section 6.1.1 of Liu et al.
-
Lecture 7 Unconstrained Optimization-Gradient Descent Method II
tl;dr: Gradient Descent for Beta Smooth Function and Convex Set
[notes]
Suggested Readings:
- Section 6.2, 2.4 and 2.5 and of Liu et al.
-
Lecture 8 Unconstrained Optimization-Gradient Descent Method III
tl;dr: Convex Function and GD for Beta-smooth and Convex Function
[notes]
Suggested Readings:
- Section 6.2, 2.4 and 2.5 and of Liu et al.
-
Lecture 9 Unconstrained Optimization-Gradient Descent Method IV
tl;dr: GD for Beta-Smooth and Alpha-Strongly Convex Function
[notes]
Suggested Readings:
- Section 6.2, 2.4 and 2.5 and of Liu et al.
-
Lecture 10 Unconstrained Optimization-Subgradient Descent
tl;dr: subgradient, subdifferential, subgradient descent
[notes]
Suggested Readings:
- Section 6.3 and 2.7 and of Liu et al.
-
Lecture 11 Unconstrained Optimization-Proximal Gradient Descent I
tl;dr: Ridge Regression, LASSO, Proximal Operator
[notes]
Suggested Readings:
- Section 5.3.2, 8.1.1 and 8.1.2 and of Liu et al.
-
Lecture 12 Unconstrained Optimization-Proximal Gradient Descent II
tl;dr: Proximal Operator, Proximal Gradient Descent
[notes]
Suggested Readings:
- Section 5.3.2, 8.1.1 and 8.1.2 and of Liu et al.
-
Lecture 13 Unconstrained Optimization-Proximal Gradient Descent III
tl;dr: Proximal Operator, Proximal Gradient Descent
[notes]
Suggested Readings:
- Section 5.3.2, 8.1.1 and 8.1.2 and of Liu et al.
-
Lecture 14 Unconstrained Optimization-Accelerated Gradient Descent
tl;dr: Accelerated Gradient Descent
[notes]
Suggested Readings:
- Section 8.2 of Liu et al.
-
Lecture 15 Unconstrained Optimization-Newton Method
tl;dr: Newton Method
[notes]
Suggested Readings:
- Section 6.4 and 6.5 of Liu et al.
-
Lecture 16 Course Review
tl;dr: Review