Machine Learning for Graphs

Course Objective

Upon completion of this course, students

• Will be acquainted with several modern approaches to apply machine learning to problems that explicitly or implicitly involve graphs (Knowledge and understanding)
• Have deep knowledge of one or more selected methods, down to the details of the primary research (Knowledge and understanding)
• Are able to implement, extend, and evaluate a selected method (Applying knowledge and understanding; Make judgements)
• Can critically reflect upon the inner workings of a selected method and on the choices made by its authors (Learning skills)
• Can engage in a scientific discussion with fellow students about the pro’s and con’s of an arbitrary method, and is able to write a scientific reports of their findings (Communication skills)

Course Content

Graphs are one of the most powerful ways of representing data. They can naturally describe a wide range of relational constructs, including social connections, historical contexts, ways of moving through a city, and interactions between molecules in a cell. Multi-relational data, consisting of entities, (multi-modal) attributes, and the labelled relations between them, can likewise be encoded as a knowledge graph in a most natural fashion. Since much of the world’s knowledge is of a multi-relational nature, the idea of knowledge graphs as the preferred data model has gained widespread adoption by technological giants such as Google and Facebook.

This widespread adoption has led to a fast growing interest in machine learning models for (knowledge) graphs, with recent years witnessing the development of various powerful deep learning techniques which enable machine learning researchers and practitioners to learn on, to learn from, and to learn with graphs directly and with minimal loss of information. These advancements have shown the potential that machine learning for graphs can bring us, while also having identified the challenges that still lie ahead.

In this course, we will focus on the bleeding edge of research on the intersection between machine learning and graphs. The course begins by revisiting the fundamentals of (spectral) graph theory as it pertains to machine learning. Next, we will focus on specific machine learning techniques for graphs, moving chronologically from established and proven methods to the more recent and cutting edge approaches, as well as discussing scalability, various downstream tasks, and learning on multi-modal heterogeneous knowledge. Finally, during the second half of the course, we will move our focus to reading and discussing recent peer-reviewed research papers.

Topics include:

• Embedding methods (Node2Vec, RDF2Vec, Distmult, RESCAL, translational methods, etc)
• Message passing methods and Graph networks (GCNs, GATs, R-GCNs, etc)
• Rule-based methods (AIMIE, AnyBURL, etc)
• Learning tasks (link prediction, node classification, representation learning, query and question answering, etc)
• Opportunities and challenges (scalability, mini-batch learning, multi-modal learning, etc)

Additional Information Teaching Methods

There will be five meetings per week, one on each day, consisting of either a (guest) lecture, a working session, or a presentation followed by a discussion. Students are assumed to not have other concurrent courses.

Lectures will be given during the first half of this course, introducing relevant background knowledge and presenting recent machine learning techniques for (knowledge) graphs. Two working sessions will be held during this period as well, providing hands-on experience with graph data and relevant machine learning tools.

During the second half of the course the format will change to a research seminar for which you need to read some of the latest literature in the field, form arguments about them, and then discuss these arguments during a scientific debate.

Students will also work on a practical project for most of the course. Weekly supervision meetings are provided, either on site or online.

The course is taught in English. Attendance during the seminar part is mandatory in order to pass.

Method of Assessment

There will be two moments of assessment:

1. Argumentative skills and participation during a discussion.
2. A scientific report, documenting the practical project.

The argumentative skills and the report are both graded, with the former counting for one-third of the final grade and the latter for two thirds. Both parts must be passed with a minimum of 5.5 to pass the course.

A resit opportunity is provided for the scientific report. There is no resit opportunity for the discussion.

Literature

There is no textbook. Reading material will be provided digitally in the form of peer-reviewed research papers from various related conferences and journals, including NIPS, AAAI, ICML, ICLR, IJCLR, ISWC, ESWC, ACL, and EMNLP.

Additional Information Target Audience

Master Artificial Intelligence (year 1)

Master Artificial Intelligence (year 2)

Custom Course Registration

Because of this intensive, participatory, and interactive seminar format, this course is limited to 60 students.

Only a registration for the main course itself is required to take part in all parts of the course.

Explanation Canvas

All course material will be made available through Canvas.

Recommended background knowledge

We require a strong background in machine learning and all topics that it implies, such as linear algebra, calculus, and probability theory. Familiarity with graph theory is recommended but not required.

An extensive experience with Python programming is a hard requirement. Practical experience with implementing machine learning models, preferably using pyTorch, is also strongly recommended. Ideally, students have taken the Machine Learning (X_400154), Data Mining Techniques (XM_40012), and/or Deep Learning course (XM_0083).