Skip to content
Technische Universität Berlin Deutsch
Course catalog and module transfer system

Information Theory and ApplicationsModule description #40981 / Version 1

← Module page Deutsche Fassung
Module / Version#40981 / #1
ValiditySince Summer semester 2020
Available language(s)English
Credit points6
Person responsible for the moduleCaire, Giuseppe
GradingGraded
Exam formatPortfolio examination
Language(s) of instructionEnglish

Affiliation

FacultyFakultät IV
InstituteInstitut für Telekommunikationssysteme
Academic chair34331600 FG Theoretische Grundlagen der Kommunikationstechnik

Contact

OfficeHFT 6
Contact personCaire, Giuseppe
E-mail addresscaire@tu-berlin.de
Websitehttp://www.commit.tu-berlin.de

Learning outcomes

At the end of this module the students will have acquired a deep knowledge of the fundamental concepts of information theory and will be able to apply such concepts to advanced applications in multiuser communication networks, channel coding, and source coding. The students will be able to analyze simple information theoretic problems by computing the entropy of a discrete source, the rate-distortion function for a continuous source with a given fidelity criterion, and the channel capacity of memoryless discrete and continuous channels. Through the lecture and the exercises, the students will apply the learned fundamental concepts to advanced problems, with applications to source coding and channel coding in a multiuser/network scenario, and will be able to implement in software and verify by computer simulation state of the art algorithms in source coding and channel coding.

Content

- Definition and main properties of entropy, cross-entropy (information divergence) and mutual information
- Fundamental Limits of data compression;
- Fundamental Limits of Data Transmission and Channel capacity;
- Fundamental limits of source compression (rate-distortion theory);
- Multiple Access Channel and Broadcast Channels as models for the uplink and downlink of wireless networks
- Coping with interference
- Relaying networks
- Project work

Module components

Compulsory area

The following courses are mandatory for this module:

CoursesTypeNumberFrequencyLanguageHours per weekCourse catalog
Information Theory and ApplicationsLecture34331600 L 011 Summer semester[en]2 ISIS
Information Theory and ApplicationsExercise34331600 L 012 Summer semester[en]2 ISIS

Workload and credit points

Information Theory and Applications (Lecture)

Workload descriptionMultiplierHoursTotal
Attendance152 h30 h
Examination preparation130 h30 h
Pre/post processing152 h30 h
90 h

Information Theory and Applications (Exercise)

Workload descriptionMultiplierHoursTotal
Attendance152 h30 h
Examination preparation130 h30 h
Pre/post processing152 h30 h
90 h

The workload of the module adds up to 180 hours. The module therefore comprises 6 credit points.

Description of teaching and learning methods

The first component of the module consists of conventional frontal teaching in class, developing theoretical and mathematical concepts, exercises developed in class, in order to develop problem-solving skills and reinforce comprehension of the theory, proposed homework exercises in order to develop independent and autonomous thinking skills in the students.

The second component of the module consists of independent reading and project work development, and preparation of a scientific presentation, training the students to develop independent thinking, spirit of initiative, and ability to present their findings in a scientific conference-like environment.

Requirements for participation and examination

Desirable prerequisites for participation in the courses:

Students must have taken the module Foundations of Stochastic Processes (40256). Special permission can be granted on a case by case basis.
In this case, students should contact directly Prof. Caire.

Mandatory requirements for the module examination registration:

This module has no examination requirements.

Completion of the module

GradingGraded
Exam formatPortfolio examination
Type of portfolio examination100 points in total
LanguageEnglish

Test elements

NamePoints/WeightCategoryDuration/Extent
(Examination) Midterm test20written60 minutes
(Examination) Final test30written60 minutes
(Deliverable assessment) Project presentation and oral discussion50oral30 minutes

Exam description

The final grade according to § 47 (2) AllgStuPO will be calculated according to Notenschlüssel 3 of Faculty IV.

Duration of the module

The following number of semesters is estimated for taking and completing the module: 1 semester(s).

This module can be started in the following semesters: Summer semester.

Maximum number of participants

This module is not limited to a number of students.

Registration procedures

The examination registration is running with Qispos.

Recommended reading, lecture notes

Lecture notes

unavailable

Electronic lecture notes

available – The script is provided in ISIS

Recommended literature

  • T. Cover and J. Thomas, Elements of Information Theory, 2nd Ed., Wiley Interscience, 2006.
  • El Gamal, Abbas, and Young-Han Kim. Network information theory. Cambridge University Press, 2011.

Assigned programs

This module version is used in the following programs:

Program / program versionSemester(s)Curriculum component
Computer Engineering
Computer Engineering (M. Sc.) - StuPO 2015
SS 2025Wahlpflicht Studiengebiete Fak. IV › Verteilte Systeme und Netze / Distributed Systems and Networks
WS 2025/26Wahlpflicht Studiengebiet B › Verteilte Systeme und Netze / Distributed Systems and Networks
SS 2026Wahlpflicht Studiengebiet B › Verteilte Systeme und Netze / Distributed Systems and Networks
WS 2026/27Wahlpflichtbereich › Studiengebiet B › Verteilte Systeme und Netze / Distributed Systems and Networks
Computer Science (Informatik)
Computer Science (Informatik) (M. Sc.) - StuPO 2015
SS 2025Wahlpflicht Studiengebiete Fak. IV › Grundlagen der Informatik / Foundations of Computing
WS 2025/26Studiengebiete › Grundlagen der Informatik / Foundations of Computing
SS 2026Studiengebiete › Verteilte Systeme und Netze / Distributed Systems and Networks
WS 2026/27Wahlpflichtbereich › Studiengebiet › Verteilte Systeme und Netze / Distributed Systems and Networks
Electrical Engineering
Electrical Engineering (M. Sc.) - StuPO 2015
SS 2025Wahlpflicht Studiengebiete Fak. IV › Kommunikationssysteme / Communication Systems
WS 2025/26Wahlpflicht Studiengebiet Kommunikationssysteme / Communication Systems › Wahlpflicht Studienrichtung Kommunikationssysteme / Communication Systems
SS 2026Wahlpflicht Studiengebiete Fak. IV › Grundlagen der Informatik / Foundations of Computing
WS 2026/27Wahlpflichtbereich › Studiengebiete Fak. IV › Grundlagen der Informatik / Foundations of Computing
Industrial Engineering and Management
Industrial Engineering and Management (M. Sc.) - StuPO 2015
SS 2020Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
WS 2020/21Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
SS 2021Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
WS 2021/22Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
SS 2022Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
WS 2022/23Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
SS 2023Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
WS 2023/24Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Communication Systems
SS 2024Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
WS 2024/25Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
SS 2025Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
WS 2025/26Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
SS 2026Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Communication Systems
WS 2026/27Ingenieurwissenschaftliche Studienrichtungen/Engineering Majors › Informations- und Kommunikationssysteme/Information and Communication Systems (IKS) › Informations- und Kommunikationssysteme/Information and Communication Systems (Wahlpflicht/Compulsory-Elective) › Distributed Systems and Networks
Industrial Engineering and Management
Industrial Engineering and Management (M. Sc.) - StuPO 2026
WS 2026/27Engineering Sciences / Specialization › Software Systems Engineering › Distributed Systems and Networks
Information Systems Management (Wirtschaftsinformatik)
Information Systems Management (Wirtschaftsinformatik) (M. Sc.) - StuPO 2017
SS 2025Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks
WS 2025/26Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks
SS 2026Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks
WS 2026/27Wahlpflichtbereich › Fachstudium Informatik › Verteilte Systeme und Netze / Distributed Systems and Networks
Technomathematics
Technomathematics (M. Sc.) - StuPO 2014
SS 2025Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET
WS 2025/26Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET
SS 2026Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET
WS 2026/27Studienrichtungen › Elektrotechnik › Elektrotechnik › Gesamtangebot Master ET
Technomathematics
Technomathematics (B. Sc.) - StuPO 2014
SS 2025Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik
WS 2025/26Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik
SS 2026Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik
WS 2026/27Studienrichtungen › Technisches Gebiet - Elektrotechnik › Technisches Gebiet - Elektrotechnik › Katalog Elektronik und Informationstechnik

Miscellaneous

Not specified