Multilingual Natural Language Processing
CMSC828I Advanced Topics in Information Processing
LING848 Seminar in Computational Linguistics
Schedule
Note that the schedule is subject to change. Students will be notified in advance of any changes that affect assignments and grading.
| Date | Topic | Readings | Due |
|---|---|---|---|
| Jan 27 | Course overview | ||
| Background: Modeling cross-lingual equivalence | |||
| Jan 29 | Aligning documents and sentences in bilingual text | Alignment, Wu (Sections 16.1--16.3). | |
| Feb 3 | Aligning words in parallel sentences (IBM Models 1, 2 and HMMs) | Statistical Machine Translation: IBM Models 1 and 2, Collins. Alignment, Wu (Section 16.4) | Pick a language to present! |
| Feb 5 | Aligning words in parallel sentences (Improved learning strategies) | Alignment by agreement, Liang et al. Discriminative Word Alignments with Conditional Random Fields, Blusom and Cohn | Reading notes |
| Feb 10 | Aligning words and constituents in parallel sentences (hierarchical models) | Alignment, Wu (Sections 16.5--16.6). Introduction to Synchronous Grammars, Chiang. | |
| Feb 12 | Finding and aligning parallel text +Amharic | Extracting Parallel Sub-Sentential Fragments from Non-Parallel Corpora, Munteanu and Marcu | Reading notes, |
| Learning and projecting syntactic representations | |||
| Feb 17 | Multilingual POS tagging Snow Day - no class! | Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging,Tackstrom et al. (2013) | Reading notes |
| Feb 19 | Multilingual syntactic parsing +Armenian,Faroese | Getting More from Morphology in Multilingual Dependency Parsing, Hohensee and Bender (2012) | Reading notes |
| Learning and projecting semantic representations | |||
| Feb 24 | Word senses and translations +Fula |
Expectations of word sense in parallel corpora, Yao et al. (2012) | Reading notes, |
| Feb 26 | Learning monolingual word representations +Georgian | Don't count, predict!, Baroni et al. (2014) | Reading notes |
| Mar 3 | Learning cross-lingual word representations +Russian | Exploiting similarities across languages for machine translation, Mikolov et al. (2013) (optional) Inducing Crosslingual Distributed Representations of Words, Klementiev et al.(2012) | Reading notes |
| Mar 5 | Snow Day - no class! | ||
| Mar 10 | Projecting predicate argument structure +Sanskrit |
Cross-lingual annotation projection of semantic roles, Pado and Lapata (2009) | Reading notes, and submit HW2 by 3/11 2pm |
| Mar 12 | How to define your course project?+Swahili | Post 1-paragraph project idea on piazza by 1pm | |
| Mar 17/19 | No class: Spring Break | ||
| Recap and Perspectives | |||
| Mar 24 | Cross-lingual projection: from morphology to discourse | Optional readings:Inducing Multilingual Text Analysis Tools via Robust Projection Across Parallel Corpora, Yarowsky et al. (2001), Cross-lingual Discourse Relation Analysis, Li et al. (2014) | |
| Mar 26 | Learning cross-lingual representations: from words to structure | Optional readings: Cross-lingual Word Clusters for Direct Transfer of Linguistic Structure, Tackstrom et al. (2012),Cross-lingual induction of semantic roles, Titov and Klementiev | Project proposal due on 3/27 |
| Mar 31 | Low-resource languages | Real-World Semi-Supervised Learning of POS-Taggers for Low-Resource Languages, Garrette et al. (2013) | Reading notes |
| Apr 2 | Low-resource languages | Read at least one of the following: Machine Translation for Language Preservation Bird and Chiang (2012) + Desperately Seeking Cebuano, Oard et al. (2003)), A Structurally diverse minimal corpus for eliciting structural mappings between languages, Probst and Lavie (2004), The Language Demographics of Amazon Mechanical Turk, Pavlick et al. (2014) | Reading notes |
| Project Background | |||
| Apr 7 | Campus Shutdown - no class! | Office hour instead | |
| Apr 9 | Cross-lingual disambiguation | Cross Lingual Pronoun Prediction (Chapter 8, p 116-134) , Hardmeier 2014. If needed, refresh your memory on maximum entropy, perceptron, gradient descent (e.g., Manning & Schuetze Chap. 16) | Reading notes |
| Apr 14 | Lab | Bring your laptop, project data and tools | |
| Apr 16 | Cross-lingual disambiguation | one of: Combining Bilingual paraphrases & Monolingual Distributional Similarity, Supervised disambiguation using parallel text | Reading notes |
| Apr 21 | Disambiguation models and algorithms: from maxent to neural networks | ||
| Apr 23 | Alignment within words | Transliteration | Reading notes |
| Apr 28 | Learning Multilingual Word Representations | Improving Vector Space Word Representations Using Multilingual Correlation | Reading notes |
| Apr 30 | Learning Multilingual Word Representations | BilBOWA: Fast Bilingual Distributed Representations without Word Alignments | Reading notes |
| May 5 | No class (DARPA PI Meeting) | work on HW#3 | HW#3 due on May 6 by 2pm |
| May 7 | Cross-lingual text classification | Cross-Language Text Classification using Structural Correspondence Learning | Reading notes |
| May 12 | Project presentations | Project deliverables due by 2pm | |
| May 20 | Take-home final exam due by 2pm | ||
Suggestions for further reading
Cross-lingual Word Clusters for Direct Transfer of Linguistic Structure, Tackstrom et al. (2012)
Inducing Multilingual Text Analysis Tools via Robust Projection Across Parallel Corpora, Yarowsky et al. (2001)
Parasense or how to use parallel corpora for word sense disambiguation, Lefever et al. (2011)
Non Parametric Bayesian Word Sense Induction, Yao and Van Durme (2011)
Learning a POS tagger from two hours of annotation, Garette and Baldridge (2013)
Learning bilingual semantic frames: shallow semantic parsing vs. semantic role projection, Fung et al. (2007)
Cross-lingual model transfer using feature representation projection, Kozhevnikov and Titov (2013).
Seven Dimensions of Portability for Language Documentation and Description, Bird and Simons (2003)
Machine Translation for Language Preservation Bird and Chiang (2012)
Neural Word Embedding as Implicit Matrix Factorization, Levy and Goldberg (2014)