Assignments

• Organize into project groups of 4-5 by 9/14
• HW 0 available here, due 9/16
• Project proposal template available here, due 10/5
• HW 1 due 10/14
• Exam 1 scheduled for 10/19 in-class
• HW 2 due 12/2
• Exam 2 scheduled for 12/9 in-class
• Project report due 12/18

Schedule

Make sure to reload this page to ensure you're seeing the latest version. Readings should be done before watching the corresponding lecture videos. See this page for materials (videos / slides / reading) from the Fall 2025 offering.

Week 1 (8/31-9/2): introduction, language modeling

• • Course introduction // [video] // [notes]
  • No associated readings!
  • HW 0 released here, due 9/16
  • Final projects:
  • • organize into groups of 4-5 by 9/14
    • proposal due 10/5 on Gradescope, use this Overleaf template


• • Language modeling // [video] // [notes]
  • [reading] Jurafsky & Martin, 3.1-3.5 (language modeling)
  • [reading] Jurafsky & Martin, 7 (neural language models)

Week 2 (9/9): neural language models

• • No class 9/7 (Labor Day)


• • Neural language models // [video] // [notes]
  • [reading] Neural language models (Bengio 2003)

Week 3 (9/14-16): Neural language models, backpropagation

• • Neural language models (cont.): decoding and composition // [video] // [notes]
  • same readings as last class


• • Neural language models (cont.): backpropagation // [video] // [notes]
  • [optional reading] Andrej Karpathy's coding-based backpropagation post
  • [optional reading] Pascanu et al., ICML 2013 (vanishing gradients in RNNs)
  • See F2020 video timestamped here for derivation of gradient through linear layer

Week 4 (9/21-23): Transformer language models

• • Attention mechanisms & self-attention // [video]
  • [reading] Bahdanau et al., ICLR 2015 (paper that introduced attention)
  • [optional reading] An easy-to-read blog post on attention


• • Transformer language models // [video]
  • [reading] Vaswani et al., NeurIPS 2017 (paper that introduced Transformers)
  • [reading] An easy-to-read blog post on Transformer language models