Quality Assessment for Machine LearningModule description #41069 / Version 1
- 3 CP
- Portfolio examination
- Graded
- Winter semester
- English
This module description is not valid in the current semester (Winter semester 2026/27).
Current version (v3)
| Module / Version | #41069 / #1 |
|---|---|
| Validity | Winter semester 2022/23 – Winter semester 2022/23 |
| Available language(s) | English, German |
| Credit points | 3 |
| Person responsible for the module | Haufe, Stefan |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Softwaretechnik und Theoretische Informatik |
| Academic chair | 34352700 FG S-Professur Unsicherheit, inverse Modellierung und maschinelles Lernen (UNIML) |
Contact
| Office | MAR 4-3 |
|---|---|
| Contact person | Haufe, Stefan |
| E-mail address | haufe@tu-berlin.de |
| Website | https://www.tu.berlin/uniml/ |
Learning outcomes
Knowledge: Students will be familiar with the requirements imposed on AI systems by current and prospective regulatory frameworks. They will know different quality dimensions of AI systems and will have elaborated procedures and metrics for measuring quality along these dimensions and technical means for assuring high quality.
Skills: Students will acquire practical skills to debug data science applications by analyzing data and/or code. In addition, students will also acquire or refine skills to independently review and systematically structure the literature of a well circumscribed field in order to address a given set of questions, and will gain experience in presenting the outcome to a critical audience as well as in participating in scientific discussions.
Competencies: Students will be able to analyze the the risks and failure modes of a given AI system or product. They will be able to suggest procedures and tests to benchmark whether a given system/product will function as intended in a variety of settings and suggest ways to overcome unwanted system behavior. Students will also be able to discuss ethical, economic and other implications of failure modes of AI systems.
Skills: Students will acquire practical skills to debug data science applications by analyzing data and/or code. In addition, students will also acquire or refine skills to independently review and systematically structure the literature of a well circumscribed field in order to address a given set of questions, and will gain experience in presenting the outcome to a critical audience as well as in participating in scientific discussions.
Competencies: Students will be able to analyze the the risks and failure modes of a given AI system or product. They will be able to suggest procedures and tests to benchmark whether a given system/product will function as intended in a variety of settings and suggest ways to overcome unwanted system behavior. Students will also be able to discuss ethical, economic and other implications of failure modes of AI systems.
Content
- motivating examples and use cases from the medical, automotive and finance domains
- taxonomy of use cases, associated risks and failure modes
- legal approaches to regulate AI systems for critical applications
- ethical considerations
- quality dimensions for AI/ML systems
* data quality
* model performance, generalization
* model robustness
* model fairness
* transparency
* uncertainty calibration
* model interpretation and types of explainability
* data privacy
- quantitative metrics and test to measure quality
- practical approaches to ensure/improve quality
- simulations and benchmarks
- current standardization efforts
- taxonomy of use cases, associated risks and failure modes
- legal approaches to regulate AI systems for critical applications
- ethical considerations
- quality dimensions for AI/ML systems
* data quality
* model performance, generalization
* model robustness
* model fairness
* transparency
* uncertainty calibration
* model interpretation and types of explainability
* data privacy
- quantitative metrics and test to measure quality
- practical approaches to ensure/improve quality
- simulations and benchmarks
- current standardization efforts
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Quality Assurance for Machine Learning | Seminar | Winter and summer semester | [en] | 2 | ISIS |
Workload and credit points
Quality Assurance for Machine Learning (Seminar)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 15 | 2 h | 30 h |
| Pre/post processing | 15 | 4 h | 60 h |
| 90 h | |||
The workload of the module adds up to 90 hours. The module therefore comprises 3 credit points.
Description of teaching and learning methods
The module currently consists of a seminar. Students will prepare a presentation to a specific topic based on a provided collection of published material. Student presentations will be framed by short lecture segments introducing, contextualizing and connecting the presented topics. Each course slot will contain discussion periods, in which active participation is fostered. Students will also conduct a group work to analyze a given AI system, and present their results to the seminar audience. Moreover, students will complete a homework.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
A BSc degree in Computer Science is recommended. Successful completion of an introductory module on ML such as "Machine Learning I" or "Machine Intelligence I" is recommended. Programming skills in at least one language (e.g., Matlab, Python, R) are required.
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points in total |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| (Deliverable assessment) Presentation | 50 | oral | 30 min |
| (Deliverable assessment) Group work | 25 | oral | 60 min |
| (Deliverable assessment) Homework | 25 | practical | 3 pages |
Exam description
The module grade is calculated based on
1. The quality of a paper presentation (50%).
2. The quality of the presentation of a group work (25%).
3. The quality of a homework (25%).
1. The quality of a paper presentation (50%).
2. The quality of the presentation of a group work (25%).
3. The quality of a homework (25%).
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Winter semester.
Maximum number of participants
The maximum capacity of participants is 25.
Registration procedures
Students can sign up for the course in MOSES.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
unavailable
Recommended literature
No recommended literature given
Assigned programs
This module is not used in any program.
Miscellaneous
Not specified