Event-based Robot Vision PROJECTModule description #41021 / Version 2
- 9 CP
- Portfolio examination
- Graded
- Winter and summer semester
- English
| Module / Version | #41021 / #2 |
|---|---|
| Validity | Since Summer semester 2025 |
| Available language(s) | English |
| Credit points | 9 |
| Person responsible for the module | Gallego, Guillermo |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Technische Informatik und Mikroelektronik |
| Academic chair | 34342000 FG Robotic Interactive Perception |
Contact
| Office | MAR 5-5 |
|---|---|
| Contact person | Gallego, Guillermo |
| E-mail address | guillermo.gallego@tu-berlin.de |
| Website | https://www.digital-future.berlin/en/about-us/professors/prof-dr-guillermo-gallego/ |
Learning outcomes
Participants of this project course will gain practical experience in applying techniques from event-based and computer vision to solve problems in robot perception (motion estimation, recognition, etc.). Participants will work individually or in a small team collaboratively, and will acquire knowledge about the state-of-the-art on event-based vision related to the chosen problem.
Content
Event-based vision is an emerging technology that promises to offer advantages to overcome some of the limitations of traditional, frame-based cameras and visual processing pipelines (from sensors to output, actionable information), such as latency, dynamic range, bandwidth and power consumption. To unlock the advantages of event-based cameras, new algorithms are needed to process their unconventional output (a stream of asynchronous pixel-wise intensity changes, as opposed to the familiar video images of standard cameras). This project is related to the investigation and development of tailored algorithms and methods to tackle specific problems in event-based vision (motion estimation, segmentation, object detection and recognition, etc.).
At the beginning of the module, students receive or select project topics from a list of possible ones, as well as some introductory material related to the chosen problem. After setting the project teams and topics, the suitable tools to carry out the project are discussed and set up. The students prepare a project plan, specify the data on which they will be working on and the steps that are anticipated for a successful completion of the project. During the remaining weeks the students develop their projects and discuss the progress with the instructor, to guide future action items. At the end of the project, the students present their findings to other students in the module, with an oral presentation. They summarize not only the technical outcome of the project but also the difficulties and lessons learned during the project.
The general topics include but are not limited to:
- Algorithms: visual odometry, SLAM, 3D reconstruction, optical flow estimation, image intensity reconstruction, recognition, stereo depth reconstruction, feature/object detection, tracking, calibration, sensor fusion (video synthesis, visual-inertial odometry, etc.).
- Event camera datasets and/or simulators.
- Event-based signal processing, representation, control, bandwidth control.
- Event-based active vision, event-based sensorimotor integration.
- Applications in: robotics (navigation, manipulation, drones...), automotive, IoT, AR/VR, space science, inspection, surveillance, crowd counting, physics, biology.
- Model-based, embedded, or learning approaches.
- Novel hardware (cameras, neuromorphic processors, etc.) and/or software platforms.
- New trends and challenges in event-based and/or biologically-inspired vision (SNNs, etc.).
- Event-based vision for computational photography.
A longer list of related topics is available in the table of content of this repository: https://github.com/uzh-rpg/event-based_vision_resources/
At the beginning of the module, students receive or select project topics from a list of possible ones, as well as some introductory material related to the chosen problem. After setting the project teams and topics, the suitable tools to carry out the project are discussed and set up. The students prepare a project plan, specify the data on which they will be working on and the steps that are anticipated for a successful completion of the project. During the remaining weeks the students develop their projects and discuss the progress with the instructor, to guide future action items. At the end of the project, the students present their findings to other students in the module, with an oral presentation. They summarize not only the technical outcome of the project but also the difficulties and lessons learned during the project.
The general topics include but are not limited to:
- Algorithms: visual odometry, SLAM, 3D reconstruction, optical flow estimation, image intensity reconstruction, recognition, stereo depth reconstruction, feature/object detection, tracking, calibration, sensor fusion (video synthesis, visual-inertial odometry, etc.).
- Event camera datasets and/or simulators.
- Event-based signal processing, representation, control, bandwidth control.
- Event-based active vision, event-based sensorimotor integration.
- Applications in: robotics (navigation, manipulation, drones...), automotive, IoT, AR/VR, space science, inspection, surveillance, crowd counting, physics, biology.
- Model-based, embedded, or learning approaches.
- Novel hardware (cameras, neuromorphic processors, etc.) and/or software platforms.
- New trends and challenges in event-based and/or biologically-inspired vision (SNNs, etc.).
- Event-based vision for computational photography.
A longer list of related topics is available in the table of content of this repository: https://github.com/uzh-rpg/event-based_vision_resources/
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Event-based Robot Vision | Project | 3434 L 11007 | Winter and summer semester | – | 6 | WS 2026/27 (1) ISIS |
Courses in the course catalog
Courses created from the teaching units of this module.
Winter semester 2026/27 (1)
Event-based Robot Vision (Project) 1
- Event-based Robot Vision (Projekt) Termingruppe 1 3434 L 11007
Workload and credit points
Event-based Robot Vision (Project)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 15 | 6 h | 90 h |
| Pre/post processing | 15 | 12 h | 180 h |
| 270 h | |||
The workload of the module adds up to 270 hours. The module therefore comprises 9 credit points.
Description of teaching and learning methods
This module contains a guided and self-organized project work, supervised by the instructor's guidance. The students receive a brief overview of the fundamentals and the recent developments in the area of event-based vision. Students work individually or in small teams on a chosen topic, and they present progress during the course as they are implementing their projects. At the end of the course, each team presents insights, methods, results and lessons learned in a final presentation. Finally, projects (code, dataset, etc.) are documented and reported in the form of a report and/or github/gitlab repository.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
Solid programming skills are required in at least one of the following programming languages: C++, Python.
Basic knowledge in robotics, machine learning and computer vision is required.
Knowledge of ROS (Robot Operating System) is desirable.
Basic knowledge in robotics, machine learning and computer vision is required.
Knowledge of ROS (Robot Operating System) is desirable.
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points in total |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| (Deliverable assessment) Final presentation | 25 | oral | 25 minutes |
| (Deliverable assessment) Implementation | 40 | practical | approx 120 hours |
| (Deliverable assessment) Intermediate presentation | 20 | oral | 20 minutes |
| (Deliverable assessment) Technical documentation | 15 | written | 8 page, repository |
Exam description
The overall grade for the module consists of the results of the course work ('portfolio exam'), with the following parts:
1. Intermediate presentation (20p): The students present their initial findings and results on their topic.
2. Implementation (40p): Code, prototype implementation of the solution to the proposed problem.
3. Final presentation (25p): The students present their final findings/results and lessons learned from the project.
4. Technical Documentation (15p): The students submit their cleaned-up code (e.g., gitlab repository), with user-friendly instructions ready to be executed by colleagues, and also prepare a short technical written scientific summary of the project.
1. Intermediate presentation (20p): The students present their initial findings and results on their topic.
2. Implementation (40p): Code, prototype implementation of the solution to the proposed problem.
3. Final presentation (25p): The students present their final findings/results and lessons learned from the project.
4. Technical Documentation (15p): The students submit their cleaned-up code (e.g., gitlab repository), with user-friendly instructions ready to be executed by colleagues, and also prepare a short technical written scientific summary of the project.
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Winter and summer semester.
Maximum number of participants
The maximum capacity of participants is 15.
Registration procedures
Within the first 4 weeks after the commencement of the project, students will have to register for the module at QISPOS (university examination protocol tool) and, additionally, at ISIS for teaching materials and communication.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
unavailable
Recommended literature
- Event-based Vision: A Survey. Gallego et al., IEEE TPAMI 2022
- Digital Image Processing, Gonzalez and Woods. Pearson
- R. Szeliski, Computer Vision: Algorithms and Applications, Springer, 2nd Edition
Assigned programs
This module version is used in the following programs:
| Program / program version | Semester(s) | Curriculum component |
|---|---|---|
| Computer Engineering Computer Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiet B › Kognitive Systeme / Cognitive Systems |
| WS 2025/26 | Wahlpflicht Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| SS 2026 | Wahlpflicht Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet B › Kognitive Systeme / Cognitive Systems | |
| Computer Science (Informatik) Computer Science (Informatik) (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems |
| WS 2025/26 | Studiengebiete › Kognitive Systeme / Cognitive Systems | |
| SS 2026 | Studiengebiete › Kognitive Systeme / Cognitive Systems | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet › Kognitive Systeme / Cognitive Systems | |
| Electrical Engineering Electrical Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems |
| WS 2025/26 | Wahlpflicht Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems | |
| SS 2026 | Wahlpflicht Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiete Fak. IV › Kognitive Systeme / Cognitive Systems | |
| Industrial Engineering and Management Industrial Engineering and Management (M. Sc.) - StuPO 2026 | WS 2026/27 | Engineering Sciences / Specialization › AI Engineering › Cognitive Systems |
| Information Systems Management (Wirtschaftsinformatik) Information Systems Management (Wirtschaftsinformatik) (M. Sc.) - StuPO 2017 | SS 2025 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems |
| WS 2025/26 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems | |
| SS 2026 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems | |
| WS 2026/27 | Wahlpflichtbereich › Fachstudium Wirtschaftsinformatik › Informationssysteme / Information Systems |
Miscellaneous
- Kognitive Systeme
- Automatisierungstechnik
- Automatisierungstechnik