Information TheoryModule description #40230 / Version 2
- 6 CP
- Portfolio examination
- Graded
- Winter and summer semester
- English
This module description is not valid in the current semester (Winter semester 2026/27).
| Module / Version | #40230 / #2 |
|---|---|
| Validity | SS 2017 – SS 2019 |
| Available language(s) | English |
| Credit points | 6 |
| Person responsible for the module | Caire, Giuseppe |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Telekommunikationssysteme |
| Academic chair | 34331600 FG Theoretische Grundlagen der Kommunikationstechnik |
Contact
| Office | HFT 6 |
|---|---|
| Contact person | Caire, Giuseppe |
| E-mail address | caire@tu-berlin.de |
| Website | http://www.commit.tu-berlin.de/menue/lehre/parameter/en/ |
Learning outcomes
Students are presented with fundamental and advanced concepts in Information Theory. The module is formed by two courses in sequence. In the first course, basic information measures and single-source/single-destination problems are presented. In the second course, multiuser network problems are presented. Information theory forms the theoretical foundation of communication networks, data compression, data storage, and source coding, such as video and audio coding.
Content
- Definition and main properties of entropy, cross-entropy (information divergence) and mutual information
- Typical sequences and typical sets, fundamental lemmas of typicality.
- Data compression, Human codes, arithmetic codes, notion of universal lossless source coding (Lempel-Ziv algorithm).
- Channel capacity of discrete memoryless channels
- Gaussian channels, spectral efficiency, waterfilling power allocation with relation to bit-loading and OFDM.
- Rate-distortion theory and quantization, reverse waterfilling with relation to subband coding, successive refinement coding, with reference to scalable image and video coding.
- The Multiple Access Channel
- The Broadcast Channel
- The Interference Channel
- Graphical networks and network coding
- Relay networks and noisy network coding
- Typical sequences and typical sets, fundamental lemmas of typicality.
- Data compression, Human codes, arithmetic codes, notion of universal lossless source coding (Lempel-Ziv algorithm).
- Channel capacity of discrete memoryless channels
- Gaussian channels, spectral efficiency, waterfilling power allocation with relation to bit-loading and OFDM.
- Rate-distortion theory and quantization, reverse waterfilling with relation to subband coding, successive refinement coding, with reference to scalable image and video coding.
- The Multiple Access Channel
- The Broadcast Channel
- The Interference Channel
- Graphical networks and network coding
- Relay networks and noisy network coding
Module components
Compulsory area
The following courses are mandatory for this module:
Workload and credit points
Information Theory (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance time | 15 | 2 h | 30 h |
| Examination preparation | 1 | 30 h | 30 h |
| Preparation and postprocessing | 15 | 2 h | 30 h |
| 90 h | |||
Network Information Theory (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Attendance time | 15 | 2 h | 30 h |
| Examination preparation | 1 | 30 h | 30 h |
| Preparation and postprocessing | 15 | 2 h | 30 h |
| 90 h | |||
The workload of the module adds up to 180 hours. The module therefore comprises 6 credit points.
Description of teaching and learning methods
The module consists of conventional frontal teaching in class, developing theoretical and mathematical concepts, exercises developed in class, in order to develop problem-solving skills and reinforce comprehension of the theory, and homework exercises in order to develop independent and autonomous thinking skills in the students.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
Prerequisite for participation to courses are a mathematical background at the level of beginning MS students in Electrical Engineering (multivariate calculus, Fourier and Laplace Transforms, signals and systems, linear algebra and notions of matrix theory). The course is open to students enrolled in any MSc in EE CS, Mathematics and Physics
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Type of portfolio examination | 100 points in total |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| (Deliverable assessment) Project presentation | 40 | oral | 20 min |
| (Examination) Oral examination/discussion | 20 | oral | 20 min |
| (Examination) Written test | 40 | written | 90 min |
Exam description
The final grade according to § 47 (2) AllgStuPO will be calculated according to Notenschlüssel 3 of Faculty
IV.
IV.
Duration of the module
The following number of semesters is estimated for taking and completing the module: 2 semester(s).
This module can be started in the following semesters: Winter and summer semester.
Maximum number of participants
This module is not limited to a number of students.
Registration procedures
Course teaching and organization (not module examination enrollment at Examination office/Prüfungsamt) is supported by an ISIS course. Registration details are provided at the beginning of the module.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
available – Course notes and slides in electronic form are provided
Recommended literature
- El Gamal, Abbas, and Young-Han Kim. Network information theory. Cambridge University Press, 2011.
- T. Cover and J. Thomas, Elements of Information Theory, 2nd Ed., Wiley Interscience, 2006.
Assigned programs
This module is not used in any program.
Miscellaneous
The module is formed by two courses, taking place in two consecutive semesters (WS - SS)