Tentative Lecture Schedule
- This schedule might change slightly during the quarter. The dates of the exam, however, will not change.
- Slides will be uploaded to the course home page, typically before each lecture. The lectures themselves might deviate significantly from the textbooks. Thus, it is necessary to attend a lecture live or view its video asynchronously to keep up with course content.
- The industry guest lectures are not included in the syllabus for the exams. But they will be the focus of the extra credit activities.
- Some topics may take a few weeks to cover.
| Week | Topic |
|---|---|
| 1-2 | Basics: Deep learning, computational graph, autodiff, ML frameworks |
| 3 | GPUs, CUDA, Communication |
| 4 | ML Compilation, graph optimizations |
| 4 | Guest lecture: TBD |
| 5 | Communication and memory optimization, distributed ML, data parallelism |
| 6 | Model parallelism, auto-parallelization |
| 7 | Transformers, LLMs, scaling law |
| 8 | LLM training, inference and serving, attention optimizations |
| 9 | Guest lecture: TBD |
| 9 | Student presentations |
| 10 | Student presentations |
| 10 | Final exam reviews |
| 11 | Final exam, date TBD |