Assignments

• HW 0 available here, due 9/17
• Project proposal template available here, due 10/6
• HW 1 available here, due 10/15
• Exam 1 scheduled for 10/20 in-class
• Exam 2 scheduled for 12/10 in-class
• Final project reports due 12/20 (last day of exams)

Schedule

Make sure to reload this page to ensure you're seeing the latest version. Readings should be done before watching the corresponding lecture videos. See this page for materials (videos / slides / reading) from the Spring 2024 offering.

Week 1 (9/3): introduction, language modeling

• • Course introduction // [video] // [notes]
  • No associated readings!
  • HW 0 released here, due 9/17
  • Final projects:
  • • organize into groups of 3-5 by 9/17
    • proposal due 10/6 on Gradescope, use this Overleaf template

Week 2 (9/7-10): language models, neural networks

• • Language modeling // [video] // [notes]
  • [reading] Jurafsky & Martin, 3.1-3.5 (language modeling)
  • [reading] Jurafsky & Martin, 7 (neural language models)
  • [S2024 video]


• • Neural language models // [video] // [notes]
  • [reading] Neural language models (Bengio 2003)
  • [S2024 video]

Week 3 (9/15-17): Neural language models, backpropagation

• • Forward propagation // [video] // [notes]
  • same readings as last class


• • Backpropagation // [video] // [notes]
  • [optional reading] Andrej Karpathy's coding-based backpropagation post
  • [optional reading] Pascanu et al., ICML 2013 (vanishing gradients in RNNs)
  • [S2024 video]
  • See F2020 video timestamped here for derivation of gradient through linear layer

Week 4 (9/22-24): Transformer language models

• • Attention mechanisms & self-attention // [video] // [notes]
  • [reading] Bahdanau et al., ICLR 2015 (paper that introduced attention)
  • [optional reading] An easy-to-read blog post on attention
  • [S2024 video]


• • Transformer language models // [video] // [notes]
  • [reading] Vaswani et al., NeurIPS 2017 (paper that introduced Transformers)
  • [reading] An easy-to-read blog post on Transformer language models
  • [S2024 video]

Week 5 (9/29-10/1): Training stages, tokenization

• • Pretraining vs. finetuning // [video] // [notes]
  • [reading] Instruction tuning (Wei et al., 2022, FLAN)
  • [reading] Nathan Lambert's slides on post-training
  • [S2024 video]


• • Tokenization // [video] // [notes]
  • [reading] Neural Machine Translation... with Subword Units (Sennrich et al., ACL 2016)
  • [reading] Catherine Arnett's blogpost on tokenizers
  • [optional reading] Byte Latent Transformer: Patches Scale Better Than Tokens (Pagnoni et al., 2025)
  • [S2024 video]

Week 6 (10/6): Position encoding & efficient attention

• • Position encoding // [video] // [notes]
  • [reading] Rotary position embeddings (RoPE, Su et al., 2021)
  • [reading] Ring attention (Liu et al., 2023)
  • [reading] Flash attention (Dao et al., 2022)
  • [S2024 video]


• • No class 10/8 (Mohit traveling)

Week 7 (10/15): Scaling laws

• • No class 10/13 (Fall break)


• • Scaling laws of LLMs // [video] // [slides]
  • [reading] Scaling Laws for Neural Language Models (Kaplan et al., 2020)
  • [reading] Training Compute-Optimal Large Language Models (Hoffmann et al., 2022)
  • [S2024 video]

Week 8 (10/20-22): In-class midterm, instruction tuning

• • 10/20: In-class midterm


• • Instruction tuning / PEFT // [video] // [notes]
  • [reading] Instruction tuning (Wei et al., 2022, FLAN)
  • [reading] LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)
  • [optionalreading] Power of Scale for Prompt Tuning (Lester et al., EMNLP 2021)

Week 9 (10/27-29): Post-training, RLHF

• • Exam 1 recap + RL intro // [video] // [notes]
  • [reading] Reinforcement learning from human feedback (Ouyang et al., 2022, RLHF)
  • [optional reading] Constitutional AI (Bai et al., 2022)
  • [optional reading] Effects of RLHF on generalization and diversity (Kirk et al., ICLR 2024)
  • [S2024 video]


• • Reward models // [video] // [notes]
  • [reading] Reinforcement learning from human feedback (Ouyang et al., 2022, RLHF)
  • [optional reading] Constitutional AI (Bai et al., 2022)
  • [optional reading] Effects of RLHF on generalization and diversity (Kirk et al., ICLR 2024)
  • [S2024 video]

Week 10 (11/3-5): RL for LLMs continued

• • REINFORCE training // [video] // [notes]


• • PPO // [video] // [notes]
  • [reading] DeepSeekMath (Shao et al., 2024, GRPO)

Week 11 (11/3-5): Reasoning models

• • GRPO and thinking // [video] // [notes]


• • Reasoning models (cont'd) // [video] // [notes]
  • [reading] s1: Simple test time scaling (Muennighoff et al., 2025)
  • [reading] DeepSeek-R1 Thoughtology (Marjanovic et al., 2025)

Week 12 (11/17-19): LLM evaluation / agents

• • Benchmarking LLMs // [video] // [notes]
  • [reading] MMLU (Hendrycks et al., ICLR 2021)
  • [reading] Judging LLM as a Judge (MTBench, Zheng et al., NeuRIPS 2023)


• • RAG and agents // [video] // [notes]
  • [reading] Retrieval-Augmented Generation (RAG, Lewis et al., NeurIPS 2020)
  • [reading] Toolformer (Schick et al., 2023)