Uncertainty in Machine LearningModule description #41113 / Version 2
- 3 CP
- Portfolio examination
- Graded
- Winter and summer semester
- English
This module description is not valid in the current semester (Winter semester 2026/27).
Current version (v3)
| Module / Version | #41113 / #2 |
|---|---|
| Validity | Winter semester 2023/24 – Winter semester 2023/24 |
| Available language(s) | English, German |
| Credit points | 3 |
| Person responsible for the module | Haufe, Stefan |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Softwaretechnik und Theoretische Informatik |
| Academic chair | 34352700 FG S-Professur Unsicherheit, inverse Modellierung und maschinelles Lernen (UNIML) |
Contact
| Office | MAR 4-4 |
|---|---|
| Contact person | Panknin, Danny |
| E-mail address | danny.panknin@tu-berlin.de |
| Website | http://tu.berlin/uniml |
Learning outcomes
Knowledge: Students have obtained an extensive overview of the manifold aspects of uncertainty in machine learning, with individual deep insights into a topic of choice.
Skills: Students have acquired or refined skills to independently review and systematically structure scientific literature and have gained experience in presenting their literature research to a critical audience as well as in participating in scientific discussions.
Competencies: Students are able to critically reflect on novel scientific results in the scope of uncertainty in machine learning. They are able to relate these results to the literature and assess their quality and impact based on adequate metrics.
Skills: Students have acquired or refined skills to independently review and systematically structure scientific literature and have gained experience in presenting their literature research to a critical audience as well as in participating in scientific discussions.
Competencies: Students are able to critically reflect on novel scientific results in the scope of uncertainty in machine learning. They are able to relate these results to the literature and assess their quality and impact based on adequate metrics.
Content
Metrology in data science and machine learning
Fundamental topics in stochastics, probability theory, and statistics
- Random variables, probability spaces, probability density functions
- Independence, marginalization
- Law of large numbers
- Central limit theorem
- Bayesian inference, priors and posteriors
- Confidence and credibility intervals
- Bootstrap, Jackknife, method of surrogate data
Uncertainty propagation through fixed measurement equations
- Guide to the Expression of Uncertainty in Measurement (GUM)
- Polynomial chaos
- Monte Carlo methods
Uncertainty estimation in Machine Learning and Deep Learning
- Aleatoric and epistemic uncertainty
- Hierarchical and empirical Bayesian models, predictive distribution
- Gaussian processes
- Errors-in-variables models
- Robust regression
- Bayesian/Probabilistic Neural Networks
- Dropout
- Ensemble Methods
- Monte Carlo methods
- Conformal prediction
- Variational inference
- Normalizing flows, invertible neural networks, diffusion models
Metrics of Uncertainty Calibration
- Proper Scoring Rules
- Calibration Curves
Advanced Topics
- Out-of-distribution detection
- Active learning for uncertainty reduction
Applications of uncertainty estimation in machine learning
Fundamental topics in stochastics, probability theory, and statistics
- Random variables, probability spaces, probability density functions
- Independence, marginalization
- Law of large numbers
- Central limit theorem
- Bayesian inference, priors and posteriors
- Confidence and credibility intervals
- Bootstrap, Jackknife, method of surrogate data
Uncertainty propagation through fixed measurement equations
- Guide to the Expression of Uncertainty in Measurement (GUM)
- Polynomial chaos
- Monte Carlo methods
Uncertainty estimation in Machine Learning and Deep Learning
- Aleatoric and epistemic uncertainty
- Hierarchical and empirical Bayesian models, predictive distribution
- Gaussian processes
- Errors-in-variables models
- Robust regression
- Bayesian/Probabilistic Neural Networks
- Dropout
- Ensemble Methods
- Monte Carlo methods
- Conformal prediction
- Variational inference
- Normalizing flows, invertible neural networks, diffusion models
Metrics of Uncertainty Calibration
- Proper Scoring Rules
- Calibration Curves
Advanced Topics
- Out-of-distribution detection
- Active learning for uncertainty reduction
Applications of uncertainty estimation in machine learning
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Uncertainty in Machine Learning | Seminar | Winter and summer semester | [en] | 2 | ISIS |
Workload and credit points
Uncertainty in Machine Learning (Seminar)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 15 | 2 h | 30 h |
| Pre/post processing | 15 | 4 h | 60 h |
| 90 h | |||
The workload of the module adds up to 90 hours. The module therefore comprises 3 credit points.
Description of teaching and learning methods
The module consists of a seminar. At the beginning of the semester, students will choose a topic based on a provided collection of published material.
During the semester, students will become acquainted with the topic and prepare a presentation, where they regularly confer with their assigned advisor.
At semester end, students will give a presentation as part of a block-seminar.
During the semester, students will become acquainted with the topic and prepare a presentation, where they regularly confer with their assigned advisor.
At semester end, students will give a presentation as part of a block-seminar.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
A BSc degree in Computer Science is recommended. Successful completion of an introductory module on ML such as "Machine Learning I" or "Machine Intelligence I" is recommended.
Basic knowledge of mathematics (particularly analysis, linear algebra and stochastics) is recommended.
Basic knowledge of mathematics (particularly analysis, linear algebra and stochastics) is recommended.
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points per element |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| (Deliverable assessment) Online Quiz | 50 | written | 45 min |
| (Deliverable assessment) Presentation | 50 | written | 30 min |
Exam description
The module grade is calculated based on
1. The quality of a paper presentation (50%).
2. The performance in an online quiz comprising all presentations of the block-seminar (50%).
1. The quality of a paper presentation (50%).
2. The performance in an online quiz comprising all presentations of the block-seminar (50%).
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Winter and summer semester.
Maximum number of participants
The maximum capacity of participants is 16.
Registration procedures
Students can sign up for the course in MOSES.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
unavailable
Recommended literature
No recommended literature given
Assigned programs
This module is not used in any program.
Miscellaneous
Not specified