Assignments

• HW 0 available here, due 9/17
• Project proposal template available here, due 10/6
• HW 1 available here, due 10/15
• Exam 1 scheduled for 10/20 in-class
• HW 2 available here, due 12/7
• Exam 2 scheduled for 12/10 in-class
• Project report template available here, due 12/20

Schedule

Make sure to reload this page to ensure you're seeing the latest version. Readings should be done before watching the corresponding lecture videos. See this page for materials (videos / slides / reading) from the Spring 2024 offering.

Week 1 (9/3): introduction, language modeling

• • Course introduction // [video] // [notes]
  • No associated readings!
  • HW 0 released here, due 9/17
  • Final projects:
  • • organize into groups of 3-5 by 9/17
    • proposal due 10/6 on Gradescope, use this Overleaf template

Week 2 (9/7-10): language models, neural networks

• • Language modeling // [video] // [notes]
  • [reading] Jurafsky & Martin, 3.1-3.5 (language modeling)
  • [reading] Jurafsky & Martin, 7 (neural language models)
  • [S2024 video]


• • Neural language models // [video] // [notes]
  • [reading] Neural language models (Bengio 2003)
  • [S2024 video]

Week 3 (9/15-17): Neural language models, backpropagation

• • Forward propagation // [video] // [notes]
  • same readings as last class


• • Backpropagation // [video] // [notes]
  • [optional reading] Andrej Karpathy's coding-based backpropagation post
  • [optional reading] Pascanu et al., ICML 2013 (vanishing gradients in RNNs)
  • [S2024 video]
  • See F2020 video timestamped here for derivation of gradient through linear layer

Week 4 (9/22-24): Transformer language models

• • Attention mechanisms & self-attention // [video] // [notes]
  • [reading] Bahdanau et al., ICLR 2015 (paper that introduced attention)
  • [optional reading] An easy-to-read blog post on attention
  • [S2024 video]


• • Transformer language models // [video] // [notes]
  • [reading] Vaswani et al., NeurIPS 2017 (paper that introduced Transformers)
  • [reading] An easy-to-read blog post on Transformer language models
  • [S2024 video]

Week 5 (9/29-10/1): Training stages, tokenization

• • Pretraining vs. finetuning // [video] // [notes]
  • [reading] Instruction tuning (Wei et al., 2022, FLAN)
  • [reading] Nathan Lambert's slides on post-training
  • [S2024 video]


• • Tokenization // [video] // [notes]
  • [reading] Neural Machine Translation... with Subword Units (Sennrich et al., ACL 2016)
  • [reading] Catherine Arnett's blogpost on tokenizers
  • [optional reading] Byte Latent Transformer: Patches Scale Better Than Tokens (Pagnoni et al., 2025)
  • [S2024 video]

Week 6 (10/6): Position encoding & efficient attention

• • Position encoding // [video] // [notes]
  • [reading] Rotary position embeddings (RoPE, Su et al., 2021)
  • [reading] Ring attention (Liu et al., 2023)
  • [reading] Flash attention (Dao et al., 2022)
  • [S2024 video]


• • No class 10/8 (Mohit traveling)

Week 7 (10/15): Scaling laws

• • No class 10/13 (Fall break)


• • Scaling laws of LLMs // [video] // [slides]
  • [reading] Scaling Laws for Neural Language Models (Kaplan et al., 2020)
  • [reading] Training Compute-Optimal Large Language Models (Hoffmann et al., 2022)
  • [S2024 video]

Week 8 (10/20-22): In-class midterm, instruction tuning

• • 10/20: In-class midterm


• • Instruction tuning / PEFT // [video] // [notes]
  • [reading] Instruction tuning (Wei et al., 2022, FLAN)
  • [reading] LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)
  • [optionalreading] Power of Scale for Prompt Tuning (Lester et al., EMNLP 2021)

Week 9 (10/27-29): Post-training, RLHF

• • Exam 1 recap + RL intro // [video] // [notes]
  • [reading] Reinforcement learning from human feedback (Ouyang et al., 2022, RLHF)
  • [optional reading] Constitutional AI (Bai et al., 2022)
  • [optional reading] Effects of RLHF on generalization and diversity (Kirk et al., ICLR 2024)
  • [S2024 video]


• • Reward models // [video] // [notes]
  • [reading] Reinforcement learning from human feedback (Ouyang et al., 2022, RLHF)
  • [optional reading] Constitutional AI (Bai et al., 2022)
  • [optional reading] Effects of RLHF on generalization and diversity (Kirk et al., ICLR 2024)
  • [S2024 video]

Week 10 (11/3-5): RL for LLMs continued

• • REINFORCE training // [video] // [notes]


• • PPO // [video] // [notes]
  • [reading] DeepSeekMath (Shao et al., 2024, GRPO)

Week 11 (11/3-5): Reasoning models

• • GRPO and thinking // [video] // [notes]


• • Reasoning models (cont'd) // [video] // [notes]
  • [reading] s1: Simple test time scaling (Muennighoff et al., 2025)
  • [reading] DeepSeek-R1 Thoughtology (Marjanovic et al., 2025)

Week 12 (11/17-19): LLM evaluation / RAG

• • Benchmarking LLMs // [video] // [notes]
  • [reading] MMLU (Hendrycks et al., ICLR 2021)
  • [reading] Judging LLM as a Judge (MTBench, Zheng et al., NeuRIPS 2023)


• • RAG and tool use // [video] // [notes]
  • [reading] Retrieval-Augmented Generation (RAG, Lewis et al., NeurIPS 2020)
  • [reading] Toolformer (Schick et al., 2023)

Week 13 (11/24): LLM agents

• • LLM agents // [video] // [notes]
  • [reading] OpenHands technical report (Wang et al.,2025)
  • [reading] OSWorld (Xie et al., 2024)
  • [optional reading] How do AI agents do human work? (Wang et al.,2025)

Week 14 (12/1-3): LLM security & memorization

• • LLM security // [video] // [notes]
  • [reading] Misalignment from reward hacking (MacDiarmid et al., 2025)
  • [codebase] "God-mode" jailbreak prompts for LLMs
  • [reading] Defending against universal jailbreaks (Sharma et al., 2025)


• • Memorization & factuality // [video] // [notes]
  • [reading] How much do language models memorize? (Morris et al., 2025)
  • [reading] FactScore (Min et al., EMNLP 2023)

Week 15 (12/8): Exam review

• • Exam review // [video] // [notes]