Lectures & Deadlines
Slides will be posted after each lecture. For exact milestone due dates and times, check Canvas.
This schedule is tentative, especially the topics of later lectures.
SLP = Speech and Language Processing by Dan Jurafsky and James H. Martin.
| # | Date | Topic | Slides | Resources | Deadlines |
|---|---|---|---|---|---|
| 1 | Sep 2 | Introduction | Christopher D. Manning. 2022. Human Language Understanding and Reasoning. Daedalus 151(2): 134–145 Chris Olah’s Research Taste Exercises Eugene Vinitsky’s Unsolicited Guide Four Categories of Acceptable Papers | - | |
| - | Sep 7 | Labor Day (no class) | - | - | - |
| 2 | Sep 9 | Text Representations | SLP ch 2, ch 5 | - | |
| 3 | Sep 14 | Language Models and RNNs | SLP ch 3, ch 14 | Project groups formed on Canvas. | |
| 4 | Sep 16 | Transformers | The Annotated Transformer The Illustrated Transformer LLMs from Scratch | - | |
| 5 | Sep 21 | Pre-training | SLP ch 7 BERT paper T5 paper GPT-3 paper | Project Brainstorming Slide due Sept 22 | |
| 6 | Sep 23 | Project Proposal Presentations | - | - | - |
| 7 | Sep 28 | Post-training | SLP ch 8, Nathan Lambert’s RLHF Book. From Sampling to Reinforce | - | |
| 8 | Sep 30 | Prompting & Fine-tuning | Scratchpads, Nye et al. 2021 CoT, Wei et al. 2023 LoRA, Hu et al. 2021 QLoRA, Dettmers et al. 2023 | Brainstorming Peer Feedback due Oct 1 | |
| 9 | Oct 5 | Generation | Speculative Decoding Inference-time Algorithms | Project Proposal due Oct 5 | |
| 10 | Oct 7 | In Class Assignment I | - | Covers up to Sept 30’s materials. | - |
| 11 | Oct 12 | Guest Lecture - Rulin Shao | - | Rulin’s website | Please fill out mid-term course evaluation! |
| 12 | Oct 14 | Guest Lecture - Sara Kangaslahti | - | Sara’s website | - |
| 13 | Oct 19 | Reasoning | Scaling Test-Time Compute Inference Scaling Laws s1 paper DeepSeek-R1 GRPO Paper | - | |
| 14 | Oct 21 | Long Contexts & Experts | Lost in the Middle FlashAttention LongFormer Switch Transformers DeepSeekMoE OLMoE | Midterm Presentation Slides due Oct 25 | |
| 15 | Oct 26 | Midterm Project Presentations | - | - | - |
| 16 | Oct 28 | Midterm Project Presentations | - | Common problems in NLP papers | - |
| 17 | Nov 2 | Evaluation & Benchmarks | HELM Crowdworker Perspectives Utility is in Eye MC Eval LLM Construct Validity | Midterm Peer Feedback due Nov 2 | |
| 18 | Nov 4 | Guest Lecture - Mayee Chen | - | Mayee’s website | - |
| 19 | Nov 9 | Guest Lecture - Nishant Balepur | - | Nishant’s website | - |
| 20 | Nov 11 | Retrieval & RAG | TBD | DPR DrQA RAG SLP ch 11 | - |
| 21 | Nov 16 | Language Agents | TBD | TBD | - |
| 22 | Nov 18 | Guest Lecture - Vijay Viswanathan | - | Vijay’s website | - |
| 23 | Nov 23 | Interpretability | TBD | TBD | - |
| 24 | Nov 25 | Guest Lecture - Caleb Ziems | - | Caleb’s website | - |
| 25 | Nov 30 | In-Class Assignment II | - | Covers Oct 5 to Nov 23’s materials. | - |
| 26 | Dec 2 | Multimodality | TBD | - | - |
| 27 | Dec 7 | Safety, Societal Impact, & Fairness | TBD | - | Final Project Report due Dec 7 |
| 28 | Dec 9 | Poster Session | - | - | Poster due Dec 9 |