Course Information

• Course: LIN 389C, Research in Computational Linguistics, 39075
• Semester: Fall 2024
• Webpage: http://jessyli.com/courses/lin389c
• Canvas: https://utexas.instructure.com/courses/1402133
• Meeting: M 9-12, RLP 4.422

Contact information

• Course organizer: Jessy Li; email: jessy@austin.utexas.edu
  • Office hours: M/W 12-1; Location: RLP 4.728

Course Purpose

To teach about, encourage, and give students time for research. Also to establish vertical cohorts among students interested in the same subfield.

Course Organization

There are six constituencies who will not be treated exactly equally in the course because their needs are different:

• first-year students: need general context and help on first research projects and first-year papers;
• second-year students: need feedback on the research they have done, prepare for further research, and write second-year papers;
• third-year students: need to write their Qualifying Papers (QP), write grant proposals;
• post-candidacy pre-proposal students: need to write and present their dissertation proposals;
• post-candidacy dissertation students: need to write dissertations, get feedback;
• students from other departments: needs will vary. NOTE that this course is designed to provide support and necessary foundations to research in Computational Linguistics, rather than an overview of research topics. You are expected to carry out one research project throughout the semester.

Course Components

• Suggested topic list (link TBA)
• Research seminar: This semester we focus on a recent papers in computational linguistics. Topics and readings will be given under schedule. For each paper, each student will prepare 3 bullets about what you like or dislike about the paper, plus questions.
• Discussion of ongoing student research:
  • Round-table: short presentations by all participants about their current research. This will happen almost every week.
  • On-going research: longer presentations (30 minutes or an hour including discussion), students, faculty, auditors if they wish.
  • Dissertation proposal presentations.
  • Dissertation progress talks.
  • Practice talks for conference presentations.

Requirements

• First-year students: Attend all classes/activities. Talk about research.
  First semester, submit literature discussion sketch halfway through the semester, submit literature discussion at end of semester.
  Second semester, submit first-year paper draft halfway through the semester, submit first-year paper at end of semester.
• Second-year students: Attend all classes/activities. Talk about research.
  First semester, submit research discussion draft halfway through the semester, submit research discussion paper at end of semester.
  Second semester, submit second-year paper draft halfway through the semester, submit second-year paper at end of semester.
• Third-year students: Attend all classes/activities. Talk about research.
  First semester, submit QP proposal halfway through the semester, submit QP progress report at end of semester.
  Second semester, submit QP draft halfway through the semester, submit QP at end of semester.
• Post-candidacy, pre-proposal students: Attend all presentations. Talk about research.
• Dissertation-writing students: Attend all presentations, give at least one presentation during semester on doctoral research.
• Students from other departments: a course project, with 2 documents: intermediate report (2-3 pages), final report (8 pages).

Schedule

Each week, unless noted otherwise, we use the first part of the class to discuss the topic of the week, and we use the second part for a round-table discussing people’s research.

• Week 1, 8/26
  • Plan for the semester: Which topics to focus on, who gives talks where.
  • Extended round table: Research results and research/publication plans for the spring.
• Week 2, 9/2 (Labor Day holiday)
• Week 3, 9/9
  • We will read Patil et al., Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence
• Week 4, 9/16
  • Julie Kallini will present her ACL 2024 best paper, Mission: Impossible Language Models
• Week 5, 9/23
  • Liu et al., Visual Instruction Tuning
• Week 6, 9/30
  • Virtual visit from Subha Pushpita and Roger Levy
• Week 7, 10/7
  • Zayne Sprague will present To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
• Week 8, 10/14
  • We will read Lightman et al., Let’s Verify Step by Step
• Week 9, 10/21
  • We will read Rafailov et al., Direct Preference Optimization: Your Language Model is Secretly a Reward Model
  • Midterm papers due
• Week 10, 10/28
  • Eva Portelance will join us and we will discuss her paper, The roles of neural networks in language acquisition
• Week 11, 11/4
  • Jackson Petty will present his work, How Does Code Pretraining Affect Language Model Task Performance?
• Week 12, 11/11
  • We will read Explaining Datasets in Words: Statistical Models with Natural Language Parameters
  • Julian Michael will visit us.
• Week 13, 11/18
  • Mario Giulianelli and Jaap Jumelet will visit us.
• Week 14, 11/25 (Thanksgiving Break)
• Week 15, 12/2
  • We will read Ethayarajh et al., KTO: Model Alignment as Prospect Theoretic Optimization
• Week 16, 12/9
  • Extended roundtable, and plans for next semester.
• Final paper deadline: 11:59pm AoE, Dec 15, 2024