[WiSe 2024/25] Seminar Quality and Usability - LLM Evaluation

Course Description

Location and Time
The seminar takes place on Mondays from 4:00 pm to 6:00 pm in person in room MAR 0.011.
Please note: There will be room changes for the first two sessions!

  • First session on 14.10.: Room MAR 0.008
  • Second session on 21.10.: Room MAR 0.003
  • All subsequent sessions: Room MAR 0.011

Online participation is not possible.

Structure
During the first three sessions, the lecturers will introduce the seminar topic.
Starting from the 4th session (11.11.2024), each session will feature presentations by students. Subsequently, each presenter will write a two-page paper on their respective topic.

The grading is weighted as follows:

  • Presentation of a topic: 40%
  • Short paper about the presented topic: 30%
  • Preparing and moderating the discussion for one session: 20%
  • Recording of a session and feedback to presenters: 10%

Seminar Content
The aim of this seminar is to develop an understanding of Large Language Models and their evaluation, as well as to practice reading, understanding, and presenting research papers. Each student will work in pairs to prepare and deliver a 40-minute presentation, moderate a discussion session, and provide feedback on another presentation. Initially, we will provide an overview of the fundamentals of Large Language Models (LLMs) and their evaluation. Subsequently, students will present on various aspects of evaluating LLMs and LLM-based applications.

Registration for the Seminar

Please read this section carefully and in its entirety. Pay attention to the two-step registration process. Furthermore, the following information is intended exclusively for Master's students. Bachelor’s students must register through the seminar meta-course.

At the first seminar session on 14.10., we will inform you about the seminar content and the implementation process. Your attendance at this session is expected.

To register for the seminar, please send an email to Yasmin Hillebrenner (yasmin.hillebrenner@tu-berlin.de) by 17.10. (11:59 pm) with the following information:

  • Name of the module (Quality and Usability) + Seminar topic (AI and Machine Learning for Dialog Systems)
  • Name
  • Student ID number
  • Study program (indicate whether Bachelor’s or Master’s)
  • Current semester
  • Registration as an elective/mandatory elective or additional module

On 18.10., all registration requests will be reviewed, and we will inform you whether you have been directly allocated a spot or are on the waiting list. If you have been granted a spot, you must register for the module in MOSES MTS by 21.10. (11:59 pm). Registrations in the MTS without prior confirmation of participation by us or Ms. Hillebrenner will not be accepted.

Waiting list spots will be allocated after 21.10. or if registrations are withdrawn.