Applied Text Mining 1: Methods

Course Objective

In this short and intensive course, students will gain hands-on experience conducting a machine learning experiment to solve an NLP task in a group work setting.

At the end of this course students will be able to independently: (1) Understand the specific needs of NLP tasks starting from literature review. (2) Preprocess and analyse existing NLP datasets and their annotations. (3) Design, set up and run machine learning experiments (developing both feature-based + transformer-based models) to solve NLP tasks (4) Evaluate and interpret the results of NLP experiments (5)Design and implement linguistically-informed error analyses for NLP tasks

Course Content

In this course students will work closely as a group to conduct an NLP experiment. This is a full-time course, where students are expected to commit ~40h/week and progressively work towards the completion of a series of NLP experiments. Each week, the different phases of the experiment will be covered (with deliverables).

Please note that the course is designed to be a practical course, and while some content will be reviewed, the students are expected to already have the necessary background to independently (as a group) conduct the NLP experiment.

During this course: (a) Students will work on understanding the task definition and formulating an experiment design. (b) Students will also work with existing datasets, understand how the data was annotated, and the need to preprocess the data and format it in a way that it can be used for the experiment; (c) Students will work on building supervised classifiers for the experiment at hand. They follow standard experimental protocols. (d) Students will be guided through reporting essential information in an NLP experiment, including: data preprocessing and statistics, general architecture of classifiers, features (if applicable), as well as evaluation and its metrics. (e) These experimental protocols will be relevant for both traditional and transformer-based classifiers and both will be part of the course. (f) Students will work on designing and implementing an error analysis for the task at hand. Students will need to understand the linguistic dimensions of the task and system’s performance. Students will analyse potential sources of error and design tests to verify their hypotheses

Additional Information Teaching Methods

This course is mostly practical. The lectures will review the basic theory related to each methodological step, which the students will apply to solve the task. A few sessions will be devoted to theoretical aspects, while other sessions will be interactive and devoted to discuss progress, doubts and problems encountered. Students will work in groups.

Students are expected to commit 40 hours per week (including both lectures and group work). Attendance is mandatory for the two weekly lectures/practical sessions.

Contact hours: 2x2 hours per week

Method of Assessment

The course includes a group+individual assignment component (with weekly deliverables) (cf. objective 2,3, 4) and an individual exam component consisting of a midterm exam (cf. objective 1) and a final (potentially) oral exam related to the weekly assignments ((objective 1,2,3,4,5). All graded components must be graded at least 5.5.

The use of AI is strongly disencouraged both for generating code and analysis. The oral exam will test ownership and understanding of all submissions.

Entry Requirements

Students must have participated in:

Introduction to Human Language Technology (L_AAMPALG016) OR Natural Language Processing (XM_0121)

AND

Machine learning for NLP (L_AAMPLIN024) OR  Machine Learning for the Quantified Self (XM_40012) OR Advanced Machine Learning (XM_0010).

Literature

The literature of the course is tied to the NLP task chosen for each run of the course. Literature will mostly comprise academic papers describing a specific NLP task, and academic papers proposing systems to address that same task.

Additional Information Target Audience

The target audience of this course is mainly students in the Language and AI track of the MA in Linguistics. Other students are welcome insofar they meet the entry requirements.

Additional Information

This is a concise course requiring 40 hours of weekly commitment. Assignments are mostly group assignments and students need to contribute to these assignments equally.

Students who are not able to commit the required time should not register for this course.

Explanation Canvas

Course materials will be made available on Canvas.

Recommended background knowledge

This course requires programming experience and the ability to independently implement and understand supervised machine learning systems (including: feature-based models such as Logistic Regression and SVMs; deep neural methods such as Bi-LSTMs; and transformer-based models such as fine-tuned BERT models).

In addition, this course also assumes prior knowledge of core linguistic concepts in morphology, syntax and semantics.