Optimization AlgorithmsModule description #41016 / Version 2
- 6 CP
- Written exam
- Graded
- Winter semester
- English
| Module / Version | #41016 / #2 |
|---|---|
| Validity | Since Winter semester 2023/24 |
| Available language(s) | English, German |
| Credit points | 6 |
| Person responsible for the module | Toussaint, Marc |
| Grading | Graded |
| Exam format | Written exam |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Technische Informatik und Mikroelektronik |
| Academic chair | 34342100 FG Intelligent Systems |
Contact
| Office | MAR 4-4 |
|---|---|
| Contact person | Toussaint, Marc |
| E-mail address | office@lis.tu-berlin.de |
| Website | https://www.user.tu-berlin.de/mtoussai/teaching/index.html |
Learning outcomes
The students will be able to develop and apply optimization algorithms.
They can formulate real-world problems appropriately as mathematical programs.
They have a detailed understanding of the different categories of optimization problems, and methods to approach them.
They have a basic understanding of the theory behind and properties of optimization algorithms.
They have an overview of and experience with existing optimization software and are able to apply them to solve optimization problems.
They can formulate real-world problems appropriately as mathematical programs.
They have a detailed understanding of the different categories of optimization problems, and methods to approach them.
They have a basic understanding of the theory behind and properties of optimization algorithms.
They have an overview of and experience with existing optimization software and are able to apply them to solve optimization problems.
Content
The course is on continuous optimization problems, with focus on non-linear mathematical programming (constrained optimization).
Part 1 introduces efficient downhill algorithms in the unconstrained case:
* gradient descent, backtracking, Wolfe conditions, convergence properties
* covariant gradient, Newton, quasi-Newton methods, BFGS
Part 2 will introduce efficient algorithms for constrained optimization:
* Basics on KKT
* Log-barriers, Augmented Lagrangian, primal-dual Newton
* Differentiable Optimization
* Convex Programs, bound constraints, Phase I
Part 3 will cover extended topics that may vary each year, e.g.:
* Stochastic Gradient Descent for NN training
* No Free Lunch, Bayesian Optimization, global optimization
* Stochastic, black-box, & evolutionary algorithms
* Existing libraries, CERES, structured NLPs, solving constraint graphs
Part 1 introduces efficient downhill algorithms in the unconstrained case:
* gradient descent, backtracking, Wolfe conditions, convergence properties
* covariant gradient, Newton, quasi-Newton methods, BFGS
Part 2 will introduce efficient algorithms for constrained optimization:
* Basics on KKT
* Log-barriers, Augmented Lagrangian, primal-dual Newton
* Differentiable Optimization
* Convex Programs, bound constraints, Phase I
Part 3 will cover extended topics that may vary each year, e.g.:
* Stochastic Gradient Descent for NN training
* No Free Lunch, Bayesian Optimization, global optimization
* Stochastic, black-box, & evolutionary algorithms
* Existing libraries, CERES, structured NLPs, solving constraint graphs
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Optimization Algorithms | Integrated course | Winter semester | [en] | 4 | WS 2026/27 (2) ISIS |
Courses in the course catalog
Courses created from the teaching units of this module.
Winter semester 2026/27 (2)
Optimization Algorithms (Integrated course) 2
Workload and credit points
Optimization Algorithms (Integrated course)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 15 | 4 h | 60 h |
| Pre/post processing | 15 | 8 h | 120 h |
| 180 h | |||
The workload of the module adds up to 180 hours. The module therefore comprises 6 credit points.
Description of teaching and learning methods
weekly lectures, exercise sessions, coding assignments and homeworks
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
Good knowledge in linear algebra and calculus
Basic programming knowledge, programming in C++ or Python
Basic programming knowledge, programming in C++ or Python
Mandatory requirements for the module examination registration:
The examination requirements could not be retrieved.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Written exam |
| Language | English |
| Duration/Extent | 120 min |
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Winter semester.
Maximum number of participants
This module is not limited to a number of students.
Registration procedures
See the ISIS course page.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
unavailable
Recommended literature
No recommended literature given
Assigned programs
This module version is used in the following programs:
| Program / program version | Semester(s) | Curriculum component |
|---|---|---|
| Computational Engineering Science (Informationstechnik im Maschinenwesen) StuPO 2018 (17.01.2018) | SS 2025 | 2. Kernbereich: Simulation und Optimierung |
| WS 2025/26 | 1. Kernbereich 1: Informatik und Mathematik | |
| SS 2026 | 1. Kernbereich 1: Informatik und Mathematik | |
| WS 2026/27 | 2. Kernbereich: Simulation und Optimierung | |
| Computer Engineering Computer Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Automatisierungstechnik / Automation and Control |
| WS 2025/26 | Wahlpflicht Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| SS 2026 | Wahlpflicht Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| Computer Science (Informatik) Computer Science (Informatik) (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems |
| WS 2025/26 | Studiengebiete › Kognitive Systeme / Cognitive Systems | |
| SS 2026 | Studiengebiete › Kognitive Systeme / Cognitive Systems | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet › Kognitive Systeme / Cognitive Systems | |
| Digital Media and Technology Digital Media and Technology (M. Sc.) - StuPO 2017 | SS 2025 | Wahlpflichtbereich › Technische Profilbereiche (A) › Profilbereich Mensch-Maschine-Interaktion |
| WS 2025/26 | Wahlpflichtbereich › Technische Profilbereiche (A) › Profilbereich Mensch-Maschine-Interaktion | |
| SS 2026 | Wahlpflichtbereich › Technische Profilbereiche (A) › Profilbereich Mensch-Maschine-Interaktion | |
| WS 2026/27 | Wahlpflichtbereich › Technische Profilbereiche (A) › Mensch-Maschine-Interaktion | |
| Electrical Engineering Electrical Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems |
| WS 2025/26 | Wahlpflicht Studiengebiete Fak. IV › Automatisierungstechnik / Automation and Control | |
| SS 2026 | Wahlpflicht Studiengebiet Automatisierungstechnik / Automation and Control › Wahlpflicht Automatisierungstechnik / Automation and Control | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems | |
| Engineering Science StuPO 2007 (19.12.2007) | SS 2025 | 2.1 Numerik und Simulation › 2.1b Ergänzungsbereich |
| WS 2025/26 | 2.1 Numerik und Simulation › 2.1b Ergänzungsbereich | |
| SS 2026 | 2.1 Numerik und Simulation › 2.1b Ergänzungsbereich | |
| Engineering Science StuPO 2020 | SS 2025 | 2.1 Numerik und Simulation › 2.1b Ergänzungsbereich |
| WS 2025/26 | 2.1 Numerik und Simulation › 2.1b Ergänzungsbereich | |
| SS 2026 | 2.1 Numerik und Simulation › 2.1b Ergänzungsbereich | |
| WS 2026/27 | 2.1 Numerik und Simulation › 2.1b Ergänzungsbereich | |
| ICT Innovation ICT Innovation (M. Sc.) - StuPO 2020 | SS 2025 | Studienrichtungen › Major: Embedded Systems › Exit year: Embedded Systems › Exit year Electives: Embedded Systems |
| WS 2025/26 | Studienrichtungen › Major: Embedded Systems › Exit year: Embedded Systems › Exit year Electives: Embedded Systems | |
| SS 2026 | Studienrichtungen › Major: Embedded Systems › Exit year: Embedded Systems › Exit year Electives: Embedded Systems | |
| Industrial Engineering and Management Industrial Engineering and Management (M. Sc.) - StuPO 2026 | WS 2026/27 | Engineering Sciences / Specialization › AI Engineering › Cognitive Systems |
| Media Technology Media Technology (M. Sc.) - StuPO 2022 | SS 2025 | Wahlpflicht weitere Profibereiche › weiterer Profilbereich Data Science |
| WS 2025/26 | Wahlpflicht weitere Profibereiche › weiterer Profilbereich Data Science | |
| SS 2026 | Wahlpflichtbereich Profilbereiche › Profilbereich Mensch-Maschine-Interaktion | |
| WS 2026/27 | Wahlpflichtbereich › Profilbereich › Mensch-Maschine-Interaktion |
Miscellaneous
Not specified