Deep Learning course - Master in Artificial Intelligence - Universitat Politècnica de Catalunya and Barcelona Supercomputing Center

THIS WEBSITE IS DEPRECATED. IT HOSTS THE CONTENTS OF THE DL COURSE FROM 2017 TO JUNE 2024. FOR THE CONTENTS FOR THE CURRENT DL COURSE AT MAI SEE RACO.

About

This is the official web page for the contents of the lectures from the Deep Learning course, at the Master in Artificial Intelligence from UPC. Here you can find basic information as well as everything needed to follow the course.

Lecturers

  • Dario Garcia-Gasulla (Course coordinator, BSC, dario.garcia@bsc.es)
  • Marc Casas (BSC, marc.casas@bsc.es)

Before starting…

This course provides access to supercomputing clusters to perform the labs. Before the first guided lab session, students must have familiarized with the environment by themselves. Instructions on how to do that will be provided timely. ALL STUDENTS must come to the first lab with a laptop able to connect to the cluster through ssh. Linux is highly recommended.

Course Structure

This course provides an applied approach to Deep Learning. It chooses to present an overview of methods and approaches, instead of going in full detail of any particular aspect. The course has three types of sessions. Theory (most content provided by lecturer often through slides), guided laboratory (content provided by lecturer to be used by students) and autonomous laboratory (work by students with support from lecturer available).

Most of the course (CNNs, RNNs, Transfer Learning and Transformers) are taught by Dario Garcia-Gasulla. The High Performance Computing (HPC) part is taught by Marc Casas. This implies that comments on each block should be addressed to the corresponding teacher. Each block may have different methodologies.

Course Calendar for the Spring 2023-2024 semester

  • (1) 16/02: Basic DL
  • (2) 23/02: CNNs
  • (3) 01/03: Lab1 Guided: CNNs
  • (4) 08/03: Lab1 Autonomous: CNNs
  • (5) 15/03: RNNs
  • (6) 22/03: Transformers
  • (7) 12/04: Transfer learning (Delivery 1st lab)
  • (8) 19/04: Lab2: Transfer learning
  • (9) 26/04: HPC
  • (10) 03/05: Lab3: HPC
  • (11) 17/05: Lab4: HPC
  • (12) 24/05: Presentations
  • (13) 31/05: Presentations

Theory

The theory part of a block provides a review of the basic concepts of Deep Learning, but is intended only as an introduction. Multiple references are given in the theory section, and the interested student should read further from those references to learn more details of the introduced topics. Beyond the cited works, there are lots of materials online of Deep Learning, although it is recommended to read more than one source, as many sources explain only one aspect or interpretation of a certain topic. A good reference for most topics is the “Deep Learning Book” by Ian Goodfellow and Yoshua Bengio and Aaron Courville. There is a physical copy of the book in the UPC library, and it can also be found online.

Guided Laboratory

The guided laboratory provides working code that can serve as a starting point for students. These codes are commented, and show a variety of algorithmic solutions. The guided laboratory will be reviewed and discussed in class with students.

Autonomous Laboratory

The autonomous laboratory session is intended for students to experiment with Deep Learning methods, and draw their own conclusions. It should also be used to obtain feedback from lecturers regarding the theoretical work in progress. For running their lab experiments, students will be given access to computational resources. However, these resources are shared with other people though a queue system. Its important to maxime the use of your resources, as the priority of your jobs will decrease as you consume your quota. Try to adjust the amount of resources you need for every job (both in number of nodes and in time). Plan your experiments in advance. Dont wait until 2 days before the delivery of the report to submit jobs, as these may be queued for a while.

Overlap with other courses

For some of the thematic blocks there is a significant amount of overlap with other courses. However, this course is oriented towards practical aspects of deep learning. This means that theory is not given in thorough detail (as other courses may do), and that a significant amount of autonomous work is expected.

Course Evaluation

The course is evaluated 25% by theory comprehension and 75% by experimental work. Theory comprehension is measured by an analysis on a paper chosen by the student and presented at the end of the semester. The student should read and fully understand the paper, reading as many references as needed for that purpose. A presentation will be done where the student will describe the paper itself, and provide constructive criticism on it. This may include, but is not limited to, answers to questions such as:

  • What is the main contribution of the article?
  • How could this paper be extended by more experiments or analysis?
  • Are there flaws in the paper methodology?
  • What future work can derive from this paper?

The 75% experimental evaluation will be based on reports for each thematic block, illustrating the conclusions derived from the laboratory sessions. Each thematic block will define the specifics of the experimental work. Alternative experimental reports suggested by the student are also acceptable, previous validation from the lecturers (e.g., replicating a particular paper results, or evaluating a different approach than the one suggested by the lecturers).

CNN / Transfer Learning lab

The first (CNNs) and second (Transfer Learning) labs are to be performed in couples. Students are free to arrange couples as they see fit. Partners can be the same for both labs or not. The evaluation of the lab will be done in a 30 minutes interview between the lecturer (Dario Garcia-Gasulla in this case) and the two students. Both students will be asked to respond separately on different aspects of the lab. This interview will be in the form of an oral report: You are expected to explain the problem you faced and the solutions you proposed, while the lecturer asks questions about the particularities of the problem, and the reasoning behing your decisions. While most of the interview will be oral, you should bring a number of supporting material (digital format is fine). These personal interviews will be scheduled during the course, shortly after the corresponding delivery date.

Materials

Code and Lab Resources

The codes used in the lab sessions can be downloaded from the following locations: