Topics

Readings

There is no required textbook. All readings will be selected from books, journal articles and conference papers that are available online freely for UMD students.

• Most required reading will be from:
  • SLP3: Speech and Language Processing (3rd edition draft) by Dan Jurfasky and James H. Martin
  • CIML: A Course in Machine Learning by Hal Daumé III
  • Seq2seq: Neural Machine Translation and Sequence-to-Sequence Models: A Tutorial by Graham Neubig
• Other recommended readings:
  • Neural Network Methods for Natural Language Processing, by Yoav Goldberg
  • Linguistics Fundamentals for NLP by Emily Bender
  • Foundations of Statistical Natural Language Processing by Chris Manning and Hinrich Schuetze

Topics

• Introduction to Disambiguation and Supervised Learning in NLP

  • Meaning of Words: Distributional Semantics, Word Sense Disambiguation
  • Supervised learning: Naive Bayes, Perceptron, Logistic Regression, Neural Networks

• Sequences

  • Language Models: N-grams, Smoothing, Recurrent Neural Networks
  • Part-of-speech Tagging
  • Structured Prediction
  • Argmax for sequences with dynamic programming

• Trees

  • Syntax and Grammars
  • Dependency Parsing, Transition-based Parsers
  • Imitation Learning

• Application: Machine Translation

  • Machine Translation and its Evaluation
  • Word alignment, IBM models, Expectation Maximization
  • Neural Sequence-to-Sequence Models, Encoder-Decoder Models, Attention
  • Selected research papers on dealing with large vocabularies, subword and character models, alternate architectures, parameter optimization, model interpretability, bias and social impact, ...