BDSPRO Big Data Systems ProjectModule description #40494 / Version 9
- 9 CP
- Portfolio examination
- Graded
- Winter and summer semester
- English
This module description is not valid in the current semester (Winter semester 2026/27).
Current version (v12)
| Module / Version | #40494 / #9 |
|---|---|
| Validity | Winter semester 2023/24 – Winter semester 2023/24 |
| Available language(s) | English |
| Credit points | 9 |
| Person responsible for the module | Markl, Volker |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Softwaretechnik und Theoretische Informatik |
| Academic chair | 34351500 FG Datenbanksysteme und Informationsmanagement (DIMA) |
Contact
| Office | FR 7-1 |
|---|---|
| Contact person | Soto, Juan |
| E-mail address | sekr@dima.tu-berlin.de |
| Website | https://www.tu.berlin/dima |
Learning outcomes
In this course students will learn how to systematically analyze a current issue in the Big Data management area as well as develop and implement a problem-oriented solution as part of a team. Students will learn how to cooperate as team member, to contribute to project organization, quality assurance and documentation, and to evaluate the quality of your solution through analysis, systematic experiments and test cases. After the course, students will be able to understand the architecture of large scale data systems and to solve problems behind the end-to-end design, implementation, and testing of large scale data management systems. They will be capable of designing and implementing solutions to improve the performance and feature-completeness of large-scale data systems in a collaborative team.
Content
Both the sciences and industry are currently undergoing a profound transformation: large-scale, diverse data sets - derived from Internet-of-Things sensors, the web, or via crowd sourcing - present a huge opportunity for data-driven decision making. This data poses new challenges in a variety of dimensions: in its unprecedented volume, in the speed at which it is generated (its velocity) and in the variety of data sources that need to be integrated. The field of Big Data Systems deals with the technological means of processing high-volume of data to gain insights from data. A whole new breed of systems and paradigms has been developed to to cope with that these challenges. However, current systems still fall short in addressing many user needs. Therefore, further system research is necessary to achieve robust query performance in any scenario. As a result, students will conduct projects that deal with topics related to current data management trends, such as Modern Hardware, Stateful Data Stream Processing, Sensor Data Management, Compiler Technology, Query Optimization, Machine Learning for Databases, and many others. This scope of the project will be adjusted to the final group size to reflect the overall workload of the course (i.e., 270h of work per student) For that, students will learn the algorithms, system design, and actual implementation of the so called Distributed Processing Platforms (e.g., Flink, NebulaStream, Spark). These are systems that execute parallel computations on terabytes of data on clusters as well as distributed Internet-of-Things topologies of up to several thousand machines. At the start of the project, a student will receive a topic as well as some information material. The team, with the assistance of the lecturer, will decide on a project environment with the suitable tools for team work, project communication, development and testing. Next, the problem will have to be analyzed, modelled and decomposed into individual components, from which tasks are derived that are subsequently assigned to smaller teams or individuals. At weekly project meetings, the project team presents progress and milestones that have been reached. In consultation with the lecturer, it is decided which further steps to take. The project is concluded with a final presentation which includes a demonstration of the prototype.
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| BDAPRO - Big Data Analytics Project | Project | 0434 L 484 | Winter and summer semester | – | 6 | WS 2026/27 (1) ISIS |
Courses in the course catalog
Courses created from the teaching units of this module.
Winter semester 2026/27 (1)
BDAPRO - Big Data Analytics Project (Project) 1
- Big Data Systems Project (BDSPRO) (Projekt) Termingruppe 1 0434 L 484
Workload and credit points
BDAPRO - Big Data Analytics Project (Project)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Documentation, Presentation | 1 | 40 h | 40 h |
| Implementation, Tests, Experiments | 1 | 130 h | 130 h |
| Participation in Meetings | 20 | 3 h | 60 h |
| Preparation Phase and Design | 1 | 40 h | 40 h |
| 270 h | |||
The workload of the module adds up to 270 hours. The module therefore comprises 9 credit points.
Description of teaching and learning methods
Guided and self-organized project work.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
Desired prerequisite knowledge and skills include:
(a) general computer science (e.g., algorithms, data structures, systems architecture, distributed systems) obtained via the completion of a Bachelor's (e.g., B.Sc. in Computer Science),
(b) mathematics (e.g., linear algebra, statistics),
(c) specific subfields in computer science (e.g., Database Technology / DBT, Data Management on Modern Hardware / DMH, Machine Learning / ML1 or MI1, Compilers),
(d) solid programming in at least one of the following programming languages: Java, C++, Scala, or Python,
(e) functional programming,
(f) basics in distributed source control management systems (i.e., Git) and software development processes such as Agile,
(g) DBT or an equivalent course on database internals prior to enrolling in BDSPRO (otherwise, be concurrently enrolled in DBT and BDSPRO in the same semester).
(a) general computer science (e.g., algorithms, data structures, systems architecture, distributed systems) obtained via the completion of a Bachelor's (e.g., B.Sc. in Computer Science),
(b) mathematics (e.g., linear algebra, statistics),
(c) specific subfields in computer science (e.g., Database Technology / DBT, Data Management on Modern Hardware / DMH, Machine Learning / ML1 or MI1, Compilers),
(d) solid programming in at least one of the following programming languages: Java, C++, Scala, or Python,
(e) functional programming,
(f) basics in distributed source control management systems (i.e., Git) and software development processes such as Agile,
(g) DBT or an equivalent course on database internals prior to enrolling in BDSPRO (otherwise, be concurrently enrolled in DBT and BDSPRO in the same semester).
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points in total |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| (Deliverable Assessment) Experiments Analysis | 20 | practical | about 30h |
| (Deliverable Assessment) Final Presentation | 20 | oral | about 20 minutes |
| (Deliverable Assessment) Intermediate Presentation | 10 | oral | about 10-15 minutes |
| (Learning Process Review) Experiment Design and Execution | 20 | practical | about 30h |
| (Learning Process Review) Prototype with Test Cases and Documentation | 30 | practical | about 60h |
Exam description
The overall grade for the module consists of the results of the course work ('portfolio exam'). The following are included in the final grade:
1. Prototype with test cases and documentation (30p.)
2. Experiment design and execution (20p.)
3. Intermediate presentation (10p.)
4. Experiments analysis (20p.)
5. Final presentation (20p.)
In the final grade, students are graded individually, i.e., final grades between students in a group can vary depending on the amount of work carried out by each person.
The final grade according to § 68 (2) AllgStuPO will be calculated with the faculty grading table 2.
(Die Gesamtnote gemäß § 68 (2) AllgStuPO wird nach dem Notenschlüssel 2 der Fakultät IV ermittelt.)
1. Prototype with test cases and documentation (30p.)
2. Experiment design and execution (20p.)
3. Intermediate presentation (10p.)
4. Experiments analysis (20p.)
5. Final presentation (20p.)
In the final grade, students are graded individually, i.e., final grades between students in a group can vary depending on the amount of work carried out by each person.
The final grade according to § 68 (2) AllgStuPO will be calculated with the faculty grading table 2.
(Die Gesamtnote gemäß § 68 (2) AllgStuPO wird nach dem Notenschlüssel 2 der Fakultät IV ermittelt.)
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Winter and summer semester.
Maximum number of participants
The maximum capacity of participants is 12.
Registration procedures
Students are required to register for the course in the official TUB examination system within six weeks after commencement of the first lecture or when the first graded assignment is due, whichever happens to be first.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
unavailable
Recommended literature
- Project specific literature will be announced in the first lecture.
Assigned programs
This module version is used in the following programs:
| Program / program version | Semester(s) | Curriculum component |
|---|---|---|
| Industrial Engineering and Management Industrial Engineering and Management (M. Sc.) - StuPO 2015 | WS 2023/24 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Data and Software Engineering |
This course is oriented towards Master's students pursuing degrees in Computer Science, Information Systems Management, Computer Engineering, or Industrial Engineering & Management.
In particular, with an interest in database systems and information management.
In particular, with an interest in database systems and information management.
Miscellaneous
Not specified