Course Information

• Course: LIN 389C, Research in Computational Linguistics, 40170
• Semester: Spring 2019
• Webpage: http://jessyli.com/courses/lin389c
• Meeting: F 12:00PM- 3:00PM, Compling Lab (RLP 4.422)

Contact information

• Course organizer: Jessy Li; email: jessy@austin.utexas.edu
  • Office hours: W 11:00-1:00, F 11:00-12:00, RLP 4.728

Course Purpose

To teach about, encourage, and give students time for research. Also to establish vertical cohorts among students interested in the same subfield.

Course Organization

There are six constituencies who will not be treated exactly equally in the course because their needs are different:

• first-year students: need general context and help on first research projects and first-year papers;
• second-year students: need feedback on the research they have done, prepare for further research, and write second-year papers;
• third-year students: need to write their Qualifying Papers (QP), write grant proposals;
• post-candidacy pre-proposal students: need to write and present their dissertation proposals;
• post-candidacy dissertation students: need to write dissertations, get feedback;
• students from other departments: needs will vary. NOTE that this course is designed to provide support and necessary foundations to research in Computational Linguistics, rather than an overview of research topics. You are expected to carry out one research project throughout the semester.

Course Components

• Suggested topic list
• Research seminar: This semester we focus on a recent papers in computational linguistics. Topics and readings will be given on the schedule page. Typically one of the students in the class will be responsible for giving a short initial summary of the paper and for preparing some questions to get the discussion going.
• Discussion of ongoing student research:
  • Round-table: short presentations by all participants about their current research. This will happen almost every week.
  • On-going research: longer presentations (30 minutes or an hour including discussion), students, faculty, auditors if they wish.
  • Dissertation proposal presentations.
  • Dissertation progress talks.
  • Practice talks for conference presentations.

Requirements

• First-year students: Attend all classes/activities. Talk about research.
  First semester, submit literature discussion sketch halfway through the semester, submit literature discussion at end of semester.
  Second semester, submit first-year paper draft halfway through the semester, submit first-year paper at end of semester.
• Second-year students: Attend all classes/activities. Talk about research.
  First semester, submit research discussion draft halfway through the semester, submit research discussion paper at end of semester.
  Second semester, submit second-year paper draft halfway through the semester, submit second-year paper at end of semester.
• Third-year students: Attend all classes/activities. Talk about research.
  First semester, submit QP proposal halfway through the semester, submit QP progress report at end of semester.
  Second semester, submit QP draft halfway through the semester, submit QP at end of semester.
• Post-candidacy, pre-proposal students: Attend all presentations. Talk about research.
• Dissertation-writing students: Attend all presentations, give at least one presentation during semester on doctoral research.
• Students from other departments: a course project, with 2 documents: intermediate report (2-3 pages), final report (8 pages).

Topics

This is the list of suggested topics we will discuss this semester (to be refined):

• Neural Turing Machines & general-purpose computing
• Dynamic topic models
• Temporal and sequence modeling
• Entity modeling
• Style transfer
• Zero-shot learning
• Bias and Interpretability of NLP models

Schedule

Each week, unless noted otherwise, we use the first part of the class to discuss the topic of the week, and we use the second part for a round-table discussing people’s research.

• Week 1, 1/25
  • Plan for the semester: Which topics to focus on, who gives talks where.
  • Extended round table: Research results and research/publication plans for the spring.
• Week 2, 2/1
  • Attention is all you need! We will discuss The Annotated Transformer. Elisa will lead the discussion.
• Week 3, 2/8
  • We will mainly read Alvarez-Melis and Jaakkola, A causal framework for explaining the predictions of black-box sequence-to-sequence models.
  • We will also read Nguyen, Comparing Automatic and Human Evaluation of Local Explanations for Text Classification as background reading.
  • Anubrata will lead the discussion.
• Week 4, 2/15
  • Yang et al., Unsupervised Text Style Transfer using Language Models as Discriminators, Laura will lead the discussion.
    • The above paper is an improvement upon Shen et al., Style Transfer from Non-Parallel Text by Cross-Alignment
• Week 5, 2/22
  • Socher et al., Zero-Shot Learning Through Cross-Modal Transfer, Angela will lead the discussion.
• Week 6, 3/1
  • Johnson et al., Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation, Daniel will lead the discussion.
• Week 7, 3/8
  • We will read Jim Burke’s notes on Introduction to Linear Optimization.
  • We will also read Cornuejols et al., A Tutorial on Integer Programming (skip the AMPL code.)
  • Pengxiang will lead the discussion.
  • Additional references:
    • IBM has some great guides for GLPK; link to the first one: Introduction to linear optimization.
    • The ILP chapter of the textbook Applied Mathematical Programming
• Week 8, 3/15
  • Ning et al., Joint Reasoning for Temporal and Causal Relations, Eric will lead the discussion.
  • Background on CCM: Chang et al., Structured learning with constrained conditional models
  • Additional references:
    • Dan Roth and Vivek Srikumar’s tutorial on ILP at EACL 2017
• Week 9, 3/22 (Spring break)
• Week 10, 3/29
  • Topic modeling overview: Computer-Assisted Text Analysis for Social Science: Topic Models and Beyond, Barea will lead the discussion.
    • Background reading:
      • Original LDA paper
      • Structured Topic Models
  • Midterm papers due midnight
• Week 11, 4/5
  • We will read Dynamic Topic Models; we will also read Momeni et al., Modeling Evolution of Topics in Large-Scale Temporal Text Corpora. Alex will lead the discussion.
    • Su’s topic modeling tutorial
    • David Blei’s variational inference tutorial
    • Time is continuous!
    • An alternative approach to DTM from Hall/Jurafsky/Manning
• Week 12, 4/12
  • We will read:
    • Cao et al., A Novel Neural Topic Model and Its Supervised Extension
    • Miao et al., Neural Variational Inference for Text Processing
    • Su will lead the discussion.
• Week 13, 4/19
  • We will read Graves et al., Neural Turing Machines; Alex will lead the discussion.
    • Fun blog post by Chris Olah
• Week 14, 4/26
  • Memory networks revisited! We will read Sukhbaatar et al., End-To-End Memory Networks.
  • We will also read Miller et al., Key-Value Memory Networks for Directly Reading Documents.
  • Alex T will lead the discussion.
• Week 15, 5/3
  • We will read about entity representations:
    • Ji et al., Dynamic Entity Representations in Neural Language Models (some of us have read this before, if that’s the case, briefly revisit the paper.)
    • Clark et al., Neural Text Generation in Stories Using Entity Representations as Context
    • Eric will lead the discussion.
• Week 16, 5/10
  • Laura will practice her talk at NLLP.
  • Elisa will practice her talk at DISRPT.
  • Roundtable.
  • Final papers due midnight