Information Theory and ApplicationsModule description #40981 / Version 1
- 6 CP
- Portfolio examination
- Graded
- Summer semester
- English
| Module / Version | #40981 / #1 |
|---|---|
| Validity | Since Summer semester 2020 |
| Available language(s) | English |
| Credit points | 6 |
| Person responsible for the module | Caire, Giuseppe |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Telekommunikationssysteme |
| Academic chair | 34331600 FG Theoretische Grundlagen der Kommunikationstechnik |
Contact
| Office | HFT 6 |
|---|---|
| Contact person | Caire, Giuseppe |
| E-mail address | caire@tu-berlin.de |
| Website | http://www.commit.tu-berlin.de |
Learning outcomes
At the end of this module the students will have acquired a deep knowledge of the fundamental concepts of information theory and will be able to apply such concepts to advanced applications in multiuser communication networks, channel coding, and source coding. The students will be able to analyze simple information theoretic problems by computing the entropy of a discrete source, the rate-distortion function for a continuous source with a given fidelity criterion, and the channel capacity of memoryless discrete and continuous channels. Through the lecture and the exercises, the students will apply the learned fundamental concepts to advanced problems, with applications to source coding and channel coding in a multiuser/network scenario, and will be able to implement in software and verify by computer simulation state of the art algorithms in source coding and channel coding.
Content
- Definition and main properties of entropy, cross-entropy (information divergence) and mutual information
- Fundamental Limits of data compression;
- Fundamental Limits of Data Transmission and Channel capacity;
- Fundamental limits of source compression (rate-distortion theory);
- Multiple Access Channel and Broadcast Channels as models for the uplink and downlink of wireless networks
- Coping with interference
- Relaying networks
- Project work
- Fundamental Limits of data compression;
- Fundamental Limits of Data Transmission and Channel capacity;
- Fundamental limits of source compression (rate-distortion theory);
- Multiple Access Channel and Broadcast Channels as models for the uplink and downlink of wireless networks
- Coping with interference
- Relaying networks
- Project work
Module components
Compulsory area
The following courses are mandatory for this module:
Workload and credit points
Information Theory and Applications (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 15 | 2 h | 30 h |
| Examination preparation | 1 | 30 h | 30 h |
| Pre/post processing | 15 | 2 h | 30 h |
| 90 h | |||
Information Theory and Applications (Exercise)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance | 15 | 2 h | 30 h |
| Examination preparation | 1 | 30 h | 30 h |
| Pre/post processing | 15 | 2 h | 30 h |
| 90 h | |||
The workload of the module adds up to 180 hours. The module therefore comprises 6 credit points.
Description of teaching and learning methods
The first component of the module consists of conventional frontal teaching in class, developing theoretical and mathematical concepts, exercises developed in class, in order to develop problem-solving skills and reinforce comprehension of the theory, proposed homework exercises in order to develop independent and autonomous thinking skills in the students.
The second component of the module consists of independent reading and project work development, and preparation of a scientific presentation, training the students to develop independent thinking, spirit of initiative, and ability to present their findings in a scientific conference-like environment.
The second component of the module consists of independent reading and project work development, and preparation of a scientific presentation, training the students to develop independent thinking, spirit of initiative, and ability to present their findings in a scientific conference-like environment.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
Students must have taken the module Foundations of Stochastic Processes (40256). Special permission can be granted on a case by case basis.
In this case, students should contact directly Prof. Caire.
In this case, students should contact directly Prof. Caire.
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points in total |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| (Examination) Midterm test | 20 | written | 60 minutes |
| (Examination) Final test | 30 | written | 60 minutes |
| (Deliverable assessment) Project presentation and oral discussion | 50 | oral | 30 minutes |
Exam description
The final grade according to § 47 (2) AllgStuPO will be calculated according to Notenschlüssel 3 of Faculty IV.
Duration of the module
The following number of semesters is estimated for taking and completing the module: 1 semester(s).
This module can be started in the following semesters: Summer semester.
Maximum number of participants
This module is not limited to a number of students.
Registration procedures
The examination registration is running with Qispos.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
available – The script is provided in ISIS
Recommended literature
- T. Cover and J. Thomas, Elements of Information Theory, 2nd Ed., Wiley Interscience, 2006.
- El Gamal, Abbas, and Young-Han Kim. Network information theory. Cambridge University Press, 2011.
Assigned programs
This module version is used in the following programs:
| Program / program version | Semester(s) | Curriculum component |
|---|---|---|
| Computer Engineering Computer Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Verteilte Systeme und Netze / Distributed Systems and Networks |
| WS 2025/26 | Wahlpflicht Studiengebiet B › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| SS 2026 | Wahlpflicht Studiengebiet B › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet B › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| Computer Science (Informatik) Computer Science (Informatik) (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Grundlagen der Informatik / Foundations of Computing |
| WS 2025/26 | Studiengebiete › Grundlagen der Informatik / Foundations of Computing | |
| SS 2026 | Studiengebiete › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiet › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| Electrical Engineering Electrical Engineering (M. Sc.) - StuPO 2015 | SS 2025 | Wahlpflicht Studiengebiete Fak. IV › Kommunikationssysteme / Communication Systems |
| WS 2025/26 | Wahlpflicht Studiengebiet Kommunikationssysteme / Communication Systems › Wahlpflicht Studienrichtung Kommunikationssysteme / Communication Systems | |
| SS 2026 | Wahlpflicht Studiengebiete Fak. IV › Grundlagen der Informatik / Foundations of Computing | |
| WS 2026/27 | Wahlpflichtbereich › Studiengebiete Fak. IV › Grundlagen der Informatik / Foundations of Computing | |
| Industrial Engineering and Management Industrial Engineering and Management (M. Sc.) - StuPO 2015 | SS 2020 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks |
| WS 2020/21 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| SS 2021 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| WS 2021/22 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| SS 2022 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| WS 2022/23 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| SS 2023 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| WS 2023/24 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Communication Systems | |
| SS 2024 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| WS 2024/25 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| SS 2025 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| WS 2025/26 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| SS 2026 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Communication Systems | |
| WS 2026/27 | Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks | |
| Industrial Engineering and Management Industrial Engineering and Management (M. Sc.) - StuPO 2026 | WS 2026/27 | Engineering Sciences / Specialization › Software Systems Engineering › Distributed Systems and Networks |
| Information Systems Management (Wirtschaftsinformatik) Information Systems Management (Wirtschaftsinformatik) (M. Sc.) - StuPO 2017 | SS 2025 | Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks |
| WS 2025/26 | Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| SS 2026 | Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| WS 2026/27 | Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks | |
| Technomathematics Technomathematics (M. Sc.) - StuPO 2014 | SS 2025 | Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET |
| WS 2025/26 | Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET | |
| SS 2026 | Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET | |
| WS 2026/27 | Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET | |
| Technomathematics Technomathematics (B. Sc.) - StuPO 2014 | SS 2025 | Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik |
| WS 2025/26 | Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik | |
| SS 2026 | Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik | |
| WS 2026/27 | Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik |
Miscellaneous
Not specified