Uncertainty in Machine LearningModule description #41113 / Version 3
- 3 CP
- Portfolio examination
- Graded
- Winter and summer semester
- English
| Module / Version | #41113 / #3 |
|---|---|
| Validity | Since Summer semester 2024 |
| Available language(s) | English, German |
| Credit points | 3 |
| Person responsible for the module | Haufe, Stefan |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Softwaretechnik und Theoretische Informatik |
| Academic chair | 34352700 FG S-Professur Unsicherheit, inverse Modellierung und maschinelles Lernen (UNIML) |
Contact
| Office | MAR 4-4 |
|---|---|
| Contact person | Panknin, Danny |
| E-mail address | danny.panknin@tu-berlin.de |
| Website | http://tu.berlin/uniml |
Learning outcomes
Knowledge: Students have obtained an extensive overview of the manifold aspects of uncertainty in machine learning, with individual deep insights into a topic of choice.
Skills: Students have acquired or refined skills to independently review and systematically structure scientific literature and have gained experience in presenting their literature research to a critical audience as well as in participating in scientific discussions.
Competencies: Students are able to critically reflect on novel scientific results in the scope of uncertainty in machine learning. They are able to relate these results to the literature and assess their quality and impact based on adequate metrics.
Skills: Students have acquired or refined skills to independently review and systematically structure scientific literature and have gained experience in presenting their literature research to a critical audience as well as in participating in scientific discussions.
Competencies: Students are able to critically reflect on novel scientific results in the scope of uncertainty in machine learning. They are able to relate these results to the literature and assess their quality and impact based on adequate metrics.
Content
Metrology in data science and machine learning
Fundamental topics in stochastics, probability theory, and statistics
- Random variables, probability spaces, probability density functions
- Independence, marginalization
- Law of large numbers
- Central limit theorem
- Bayesian inference, priors and posteriors
- Confidence and credibility intervals
- Bootstrap, Jackknife, method of surrogate data
Uncertainty propagation through fixed measurement equations
- Guide to the Expression of Uncertainty in Measurement (GUM)
- Polynomial chaos
- Monte Carlo methods
Uncertainty estimation in Machine Learning and Deep Learning
- Aleatoric and epistemic uncertainty
- Hierarchical and empirical Bayesian models, predictive distribution
- Gaussian processes
- Errors-in-variables models
- Robust regression
- Bayesian/Probabilistic Neural Networks
- Dropout
- Ensemble Methods
- Monte Carlo methods
- Conformal prediction
- Variational inference
- Normalizing flows, invertible neural networks, diffusion models
Metrics of Uncertainty Calibration
- Proper Scoring Rules
- Calibration Curves
Advanced Topics
- Out-of-distribution detection
- Active learning for uncertainty reduction
Applications of uncertainty estimation in machine learning
Fundamental topics in stochastics, probability theory, and statistics
- Random variables, probability spaces, probability density functions
- Independence, marginalization
- Law of large numbers
- Central limit theorem
- Bayesian inference, priors and posteriors
- Confidence and credibility intervals
- Bootstrap, Jackknife, method of surrogate data
Uncertainty propagation through fixed measurement equations
- Guide to the Expression of Uncertainty in Measurement (GUM)
- Polynomial chaos
- Monte Carlo methods
Uncertainty estimation in Machine Learning and Deep Learning
- Aleatoric and epistemic uncertainty
- Hierarchical and empirical Bayesian models, predictive distribution
- Gaussian processes
- Errors-in-variables models
- Robust regression
- Bayesian/Probabilistic Neural Networks
- Dropout
- Ensemble Methods
- Monte Carlo methods
- Conformal prediction
- Variational inference
- Normalizing flows, invertible neural networks, diffusion models
Metrics of Uncertainty Calibration
- Proper Scoring Rules
- Calibration Curves
Advanced Topics
- Out-of-distribution detection
- Active learning for uncertainty reduction
Applications of uncertainty estimation in machine learning
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Uncertainty in Machine Learning | Seminar | Winter and summer semester | [en] | 2 | ISIS |
Workload and credit points
Uncertainty in Machine Learning (Seminar)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 15 | 2 h | 30 h |
| Pre/post processing | 15 | 4 h | 60 h |
| 90 h | |||
The workload of the module adds up to 90 hours. The module therefore comprises 3 credit points.
Description of teaching and learning methods
The module consists of a seminar, which will either be held in a weekly format or as a block seminar over two full days at the end of the semester.
At the beginning of the semester, students will choose a topic based on a provided collection of published material.
Students will become acquainted with the topic and prepare a presentation to be held in class, where they regularly confer with their assigned advisor.
The course grade is composed of the student's presentation and a quiz that comprises all presentations of the seminar.
At the beginning of the semester, students will choose a topic based on a provided collection of published material.
Students will become acquainted with the topic and prepare a presentation to be held in class, where they regularly confer with their assigned advisor.
The course grade is composed of the student's presentation and a quiz that comprises all presentations of the seminar.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
A BSc degree in Computer Science is recommended. Successful completion of an introductory module on ML such as "Machine Learning I" or "Machine Intelligence I" is recommended.
Basic knowledge of mathematics (particularly analysis, linear algebra and stochastics) is recommended.
Basic knowledge of mathematics (particularly analysis, linear algebra and stochastics) is recommended.
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points per element |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| (Deliverable assessment) Online Quiz | 50 | written | 45 min |
| (Deliverable assessment) Presentation | 50 | written | 30 min |
Exam description
The module grade is calculated based on
1. The quality of a paper presentation (50%).
2. The performance in an online quiz comprising all presentations of the seminar (50%).
1. The quality of a paper presentation (50%).
2. The performance in an online quiz comprising all presentations of the seminar (50%).
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Winter and summer semester.
Maximum number of participants
The maximum capacity of participants is 16.
Registration procedures
Students can sign up for the course in MOSES.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
unavailable
Recommended literature
No recommended literature given
Assigned programs
This module version is used in the following programs:
| Program / program version | Semester(s) | Curriculum component |
|---|---|---|
| Computer Engineering Computer Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Medientechnik und Mensch-Maschine-Interaktion / Digital Media and Human-Computer Interaction |
| WS 2025/26 | Wahlpflicht Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| SS 2026 | Wahlpflicht Studiengebiet B › Medientechnik und Mensch-Maschine-Interaktion / Digital Media und Human-Computer Interaction | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| Computer Science (Informatik) Computer Science (Informatik) (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Medientechnik und Mensch-Maschine-Interaktion / Digital Media and Human-Computer Interaction |
| WS 2025/26 | Studiengebiete › Kognitive Systeme / Cognitive Systems | |
| SS 2026 | Studiengebiete › Kognitive Systeme / Cognitive Systems | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet › Kognitive Systeme / Cognitive Systems | |
| Electrical Engineering Electrical Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Medientechnik und Mensch-Maschine-Interaktion / Digital Media and Human-Computer Interaction |
| WS 2025/26 | Wahlpflicht Studiengebiete Fak. IV › Medientechnik und Mensch-Maschine-Interaktion / Digital Media and Human-Computer Interaction | |
| SS 2026 | Wahlpflicht Studiengebiete Fak. IV › Medientechnik und Mensch-Maschine-Interaktion / Digital Media and Human-Computer Interaction | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiete Fak. IV › Medientechnik und Mensch-Maschine-Interaktion / Digital Media and Human-Computer Interaction | |
| Industrial Engineering and Management Industrial Engineering and Management (M. Sc.) - StuPO 2026 | WS 2026/27 | Engineering Sciences / Specialization › AI Engineering › Cognitive Systems |
| Information Systems Management (Wirtschaftsinformatik) Information Systems Management (Wirtschaftsinformatik) (M. Sc.) - StuPO 2017 | SS 2025 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems |
| WS 2025/26 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems | |
| SS 2026 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems | |
| WS 2026/27 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems |
Miscellaneous
Not specified