Information Theory and Coding (ET)Module description #40101 / Version 2
- 12 CP
- Portfolio examination
- Graded
- Winter and summer semester
- English
This module description is not valid in the current semester (Winter semester 2026/27).
| Module / Version | #40101 / #2 |
|---|---|
| Validity | SS 2015 – SS 2015 |
| Available language(s) | English |
| Credit points | 12 |
| Person responsible for the module | Caire, Giuseppe |
| Grading | Graded |
| Exam format | Portfolio examination |
| Language(s) of instruction | English |
Affiliation
| Faculty | Fakultät IV |
|---|---|
| Institute | Institut für Telekommunikationssysteme |
| Academic chair | 34331600 FG Theoretische Grundlagen der Kommunikationstechnik |
Contact
| Office | HFT 6 |
|---|---|
| Contact person | Caire, Giuseppe |
| E-mail address | caire@tu-berlin.de |
| Website | http://www.commit.tu-berlin.de/menue/lehre/ |
Learning outcomes
Communication networks are perhaps the most pervasive and ubiquitous infrastructure of our
times. Billions of people around the world access information, make business, share experiences and stay
in touch with each other through the Internet, and in particular using wireless access.
The fundamental science and engineering knowledge that made this astonishing development
possible finds its origin in the seminal paper by Claude E. Shannon, "A Mathematical Theory of Communication", published in 1948. This fundamental work represents the birth of the modern
information age. Astonishingly, at a time where the telephone networks was
analog and based on electromechanical circuit switching and radio reduced to analog broadcasting,
Shannon deployed the fundamental concepts of information measures (Entropy, Mutual Information)
and their operational significance as the fundamental limits of data compression (how many
bits of information are necessary to compress an information source without any loss?), channel
coding (how many bits of information per unit time can be sent through a communication channel
such that the message is recovered by the receiver with arbitrarily low error probability?) and
source coding (how many bits of information are necessary to represent an information source such
that its can be reconstructed within a desired fidelity level?).
Since Shannon's work, the field of modern digital communications, channel coding, data networks,
source coding and wireless/cellular communications has developed enormously. Very large
and successful companies, such as Qualcomm, Ericsson, Nokia, Alcatel Lucent, Siemens, Huawei,
Broadcom, Intel, Samsung, have been setting the technology trends, and standards such as TCP/IP,
IEEE 802.11, IEEE 802.16, GSM, IS-95, and the more recent 3GPP family of standards (e.g.,
LTE/LTE-A) enable ubiquitous connectivity and the merging of a variety of communication networks
onto a common interconnected platform. Perhaps the most striking fact is that Shannon's
theoretical conclusions, that information is essentially "discrete" (i.e., digital) and that any source
can be reduced to a common currency (bits), such bits can be exchanged reliably through a network
which is somehow source-agnostic, and reconstructed at the end users within a near-optimal fidelity
level, has become a fact of everyday's life, that we all enjoy and experience. Today, we use the
Internet, most of the times supported by a wireless access network, to consume a wealth of different
sources (audio, speech, video, data, images, text) that in the not-so-distant past required separate
and very different networks (e.g., FM radio, land-line telephone networks, cellular telephone
networks, telegraph/telex, mail/currier delivery, cable TV, UHF/VHF TV, Satellite TV).
The module ``Information Theory and Coding'' provides a comprehensive background on
foundations and specific theoretical knowledge on information theory and channel coding, thus providing the
theoretical background that stays at the basis of modern digital communication networks. Furthermore,
through a rich set of possible elective courses, the students will have the opportunity of
acquiring additional knowledge in specific areas of interest, some of which are advanced theoretical, and others are more applied and practical, according to each student specific orientation and interest.
times. Billions of people around the world access information, make business, share experiences and stay
in touch with each other through the Internet, and in particular using wireless access.
The fundamental science and engineering knowledge that made this astonishing development
possible finds its origin in the seminal paper by Claude E. Shannon, "A Mathematical Theory of Communication", published in 1948. This fundamental work represents the birth of the modern
information age. Astonishingly, at a time where the telephone networks was
analog and based on electromechanical circuit switching and radio reduced to analog broadcasting,
Shannon deployed the fundamental concepts of information measures (Entropy, Mutual Information)
and their operational significance as the fundamental limits of data compression (how many
bits of information are necessary to compress an information source without any loss?), channel
coding (how many bits of information per unit time can be sent through a communication channel
such that the message is recovered by the receiver with arbitrarily low error probability?) and
source coding (how many bits of information are necessary to represent an information source such
that its can be reconstructed within a desired fidelity level?).
Since Shannon's work, the field of modern digital communications, channel coding, data networks,
source coding and wireless/cellular communications has developed enormously. Very large
and successful companies, such as Qualcomm, Ericsson, Nokia, Alcatel Lucent, Siemens, Huawei,
Broadcom, Intel, Samsung, have been setting the technology trends, and standards such as TCP/IP,
IEEE 802.11, IEEE 802.16, GSM, IS-95, and the more recent 3GPP family of standards (e.g.,
LTE/LTE-A) enable ubiquitous connectivity and the merging of a variety of communication networks
onto a common interconnected platform. Perhaps the most striking fact is that Shannon's
theoretical conclusions, that information is essentially "discrete" (i.e., digital) and that any source
can be reduced to a common currency (bits), such bits can be exchanged reliably through a network
which is somehow source-agnostic, and reconstructed at the end users within a near-optimal fidelity
level, has become a fact of everyday's life, that we all enjoy and experience. Today, we use the
Internet, most of the times supported by a wireless access network, to consume a wealth of different
sources (audio, speech, video, data, images, text) that in the not-so-distant past required separate
and very different networks (e.g., FM radio, land-line telephone networks, cellular telephone
networks, telegraph/telex, mail/currier delivery, cable TV, UHF/VHF TV, Satellite TV).
The module ``Information Theory and Coding'' provides a comprehensive background on
foundations and specific theoretical knowledge on information theory and channel coding, thus providing the
theoretical background that stays at the basis of modern digital communication networks. Furthermore,
through a rich set of possible elective courses, the students will have the opportunity of
acquiring additional knowledge in specific areas of interest, some of which are advanced theoretical, and others are more applied and practical, according to each student specific orientation and interest.
Content
The module ``Information Theory and Coding'' consists of 4 courses of 3 Pts each, for a total of
12 Pts (ECTS). Two of these courses are required:
1) Information Theory;
2) Modern Channel Coding;
Other two can be chosen within a list of elective courses.
Information Theory presents the fundamentals of Information Theory. In particular,
information measures and information sources, and the fundamental limits of data compression,
channel coding and (lossy) source coding. Relations to channel coding theory, communication
theory and signal processing (notably, audio, image and video coding) are evidenced.
Modern Channel Coding presents an overview of the channel coding techniques currently
used in wireless network standards and data storage systems. Classical coding theory (algebraic
coding) is presented at a glance, with some examples of classical codes (Hamming codes and their
duals) and particular emphasis on Reed-Solomon codes. Then, a graph-theoretic framework is
adopted to present convolutional codes and sparse graph codes (Turbo Codes, LDPC codes) and
their decoding algorithms (Viterbi Decoding, Belief Propagation Decoding). Finally, some special
topics are presented as on-going research, such as regenerative storage codes and their relationship
with network coding, spatially coupled codes and their iterative decoding threshold analysis.
In the list of elective courses, there is the possibility of choosing the two lab-oriented courses in a
sequence: Wireless Digital Communications Lab I and Wireless Digital Communications Lab II, which
provide the students with a hands-on practical training based on Matlab, on the
algorithms studied in theory on the two required courses. Other elective include:
Foundations of Digital and Wireless Communications (basic principles
of communication over time-varying wireless channels, performance measures, stochastic modeling of
wireless channels), Estimation and Decision Theory for Communication Systems (providing the fundamentals of statistical signal processing applied to
communication problems), Projection Methods in Signal Processing and Communications (providing a
theoretical framework for optimization in vector spaces, essential for signal processing and communication
algorithms), Compressed Sensing (also known as sparse signal reconstruction, provides the basics of a
new area in signal processing focused on the estimation of inherently redundant signals when only a small
number of linear projections are available, which is gaining more and more importance in areas such as
image coding, features identification, machine learning schemes, channel estimation in wireless
communications, and computation in big-data sets), Network Information Theory (this presents the
extension of classical information theory to network problems, i.e., problems involving more than one
source and one destination, which is at the basis of modern and efficient system design in wireless
networks),
Modern Channel Coding (a comprehensive overview of ``graph-based'' modern channel codes and their
iterative decoding algorithms, such as Turbo Codes, LDPC codes, Spatially coupled LDPC codes),
Cross-Layer Optimization and Resource Allocation in Wireless Networks (methods to formulate and solve
optimization problems related to wireless networks, including decentralized solutions suitable to on-line
protocol implementation), Wireless Communication Systems (a deep overview of present and future
wireless communication standards, their design rationale and their main working principles), Physical-
Layer Security (information-theoretic methods to achieve secure communication over a wireless channel),
MIMO Systems and Adaptive Transmission (schemes and techniques for multiple-antenna systems, which
have become very popular in recent years and have been massively adopted in wireless communication
standards such as IEEE 802.11n, 802.11ac, and 3GPP LTE-Advanced).
12 Pts (ECTS). Two of these courses are required:
1) Information Theory;
2) Modern Channel Coding;
Other two can be chosen within a list of elective courses.
Information Theory presents the fundamentals of Information Theory. In particular,
information measures and information sources, and the fundamental limits of data compression,
channel coding and (lossy) source coding. Relations to channel coding theory, communication
theory and signal processing (notably, audio, image and video coding) are evidenced.
Modern Channel Coding presents an overview of the channel coding techniques currently
used in wireless network standards and data storage systems. Classical coding theory (algebraic
coding) is presented at a glance, with some examples of classical codes (Hamming codes and their
duals) and particular emphasis on Reed-Solomon codes. Then, a graph-theoretic framework is
adopted to present convolutional codes and sparse graph codes (Turbo Codes, LDPC codes) and
their decoding algorithms (Viterbi Decoding, Belief Propagation Decoding). Finally, some special
topics are presented as on-going research, such as regenerative storage codes and their relationship
with network coding, spatially coupled codes and their iterative decoding threshold analysis.
In the list of elective courses, there is the possibility of choosing the two lab-oriented courses in a
sequence: Wireless Digital Communications Lab I and Wireless Digital Communications Lab II, which
provide the students with a hands-on practical training based on Matlab, on the
algorithms studied in theory on the two required courses. Other elective include:
Foundations of Digital and Wireless Communications (basic principles
of communication over time-varying wireless channels, performance measures, stochastic modeling of
wireless channels), Estimation and Decision Theory for Communication Systems (providing the fundamentals of statistical signal processing applied to
communication problems), Projection Methods in Signal Processing and Communications (providing a
theoretical framework for optimization in vector spaces, essential for signal processing and communication
algorithms), Compressed Sensing (also known as sparse signal reconstruction, provides the basics of a
new area in signal processing focused on the estimation of inherently redundant signals when only a small
number of linear projections are available, which is gaining more and more importance in areas such as
image coding, features identification, machine learning schemes, channel estimation in wireless
communications, and computation in big-data sets), Network Information Theory (this presents the
extension of classical information theory to network problems, i.e., problems involving more than one
source and one destination, which is at the basis of modern and efficient system design in wireless
networks),
Modern Channel Coding (a comprehensive overview of ``graph-based'' modern channel codes and their
iterative decoding algorithms, such as Turbo Codes, LDPC codes, Spatially coupled LDPC codes),
Cross-Layer Optimization and Resource Allocation in Wireless Networks (methods to formulate and solve
optimization problems related to wireless networks, including decentralized solutions suitable to on-line
protocol implementation), Wireless Communication Systems (a deep overview of present and future
wireless communication standards, their design rationale and their main working principles), Physical-
Layer Security (information-theoretic methods to achieve secure communication over a wireless channel),
MIMO Systems and Adaptive Transmission (schemes and techniques for multiple-antenna systems, which
have become very popular in recent years and have been massively adopted in wireless communication
standards such as IEEE 802.11n, 802.11ac, and 3GPP LTE-Advanced).
Module components
Compulsory area
The following courses are mandatory for this module:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Information Theory | Lecture | 0432 L 654 | Winter semester | – | 2 | ISIS |
| Modern Channel Coding | Lecture | Summer semester | – | 2 | ISIS |
Compulsory elective area
A selection of the following courses must be completed:
| Courses | Type | Number | Frequency | Language | Hours per week | Course catalog |
|---|---|---|---|---|---|---|
| Compressed Sensing | Lecture | 0432 L 664 | Winter semester | – | 2 | ISIS |
| Cross-Layer Optimization and Resource Allocation in Wireless Networks | Lecture | 34331600 L 004 | Summer semester | – | 2 | ISIS |
| Estimation and Decision Theory for Communication Systems | Lecture | 0432 L 630 | Winter semester | – | 2 | ISIS |
| Foundations of Digital and Wireless Communications | Lecture | 0432 L 657 | Winter semester | – | 2 | ISIS |
| MIMO Systems and Adaptive Transmission | Lecture | 34331600 L 006 | Summer semester | – | 2 | ISIS |
| Network Information Theory | Lecture | 34331600 L 001 | Summer semester | – | 2 | ISIS |
| Physical-Layer Security | Lecture | 34331600 L 002 | Summer semester | – | 2 | ISIS |
| Projection methods in modern signal processing and communications | Lecture | 0432 L 649 | Winter semester | – | 2 | ISIS |
| Wireless Communication Systems | Lecture | 0432 L 615 | Summer semester | – | 2 | WS 2026/27 (1) ISIS |
| Wireless Digital Communications Lab I | Internship | 0432 L 658 | Winter semester | – | 2 | WS 2026/27 (1) ISIS |
| Wireless Digital Communications Lab II | Internship | 34331600 L 005 | Summer semester | – | 2 | ISIS |
Courses in the course catalog
Courses created from the teaching units of this module.
Winter semester 2026/27 (2)
Wireless Communication Systems (Lecture) 1
- Wireless Communication Systems (Vorlesung) Termingruppe 1 0432 L 615
Wireless Digital Communications Lab I (Internship) 1
- Wireless Digital Communications Lab I (Praktikum) 1. Termingruppe 0432 L 658
Workload and credit points
Information Theory (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Modern Channel Coding (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Compressed Sensing (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Cross-Layer Optimization and Resource Allocation in Wireless Networks (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Estimation and Decision Theory for Communication Systems (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Foundations of Digital and Wireless Communications (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
MIMO Systems and Adaptive Transmission (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Network Information Theory (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Physical-Layer Security (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Projection methods in modern signal processing and communications (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Wireless Communication Systems (Lecture)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Wireless Digital Communications Lab I (Internship)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
Wireless Digital Communications Lab II (Internship)
| Workload description | Multiplier | Hours | Total |
|---|---|---|---|
| Präsenzzeit | 15 | 2 h | 30 h |
| Vor-/Nachbereitung | 15 | 4 h | 60 h |
| 90 h | |||
The workload of the module adds up to 1170 hours. The module therefore comprises 12 credit points.
Description of teaching and learning methods
Courses are taught through lectures, problem-solving sessions (exercises in class),
and integrated laboratory practical work, computer projects (mainly in Matlab), and homework/independent
work at home. Attendance is required in all events held in the classroom and in the labs, both practical
work and lectures.
and integrated laboratory practical work, computer projects (mainly in Matlab), and homework/independent
work at home. Attendance is required in all events held in the classroom and in the labs, both practical
work and lectures.
Requirements for participation and examination
Desirable prerequisites for participation in the courses:
Suitable desired pre-requisites for participation to courses are a basic
knowledge of information technology, as described in particular in the courses "Signale und Systeme"
and "Nachrichtenübertragung" of the field of study "Elektronik und Informationstechnik"
of the BSc degree program. This module is open to students enrolled in the
Elektrotechnik MSc degree program. For participation in the laboratory work,
basic knowledge of programming with MatLab is desirable, but it will be in any case provided for those
students who do not have such background.
knowledge of information technology, as described in particular in the courses "Signale und Systeme"
and "Nachrichtenübertragung" of the field of study "Elektronik und Informationstechnik"
of the BSc degree program. This module is open to students enrolled in the
Elektrotechnik MSc degree program. For participation in the laboratory work,
basic knowledge of programming with MatLab is desirable, but it will be in any case provided for those
students who do not have such background.
Mandatory requirements for the module examination registration:
This module has no examination requirements.
Completion of the module
| Grading | Graded |
|---|---|
| Exam format | Portfolio examination |
| Language | English |
Test elements
| Name | Points/Weight | Category | Duration/Extent |
|---|---|---|---|
| Oral performance assessments, including possible project presentations, for all elective courses | 20 | ||
| Written performance assessments for the required courses | 30 |
Exam description
A total of 100 points portfolio can be achieved:
• two written performance assessments (30 portfolio points each) for the required courses:
Foundations of Digital and Wireless Communications and Information Theory.
• Oral performance assessments, including possible project presentations, for all other elective courses (20
portfolio points each).
Die Gesamtnote gemäß § 47 (2) AllgStuPO wird nach dem Notenschlussel 3 der Fakultät IV ermittelt
• two written performance assessments (30 portfolio points each) for the required courses:
Foundations of Digital and Wireless Communications and Information Theory.
• Oral performance assessments, including possible project presentations, for all other elective courses (20
portfolio points each).
Die Gesamtnote gemäß § 47 (2) AllgStuPO wird nach dem Notenschlussel 3 der Fakultät IV ermittelt
Duration of the module
The following number of semesters is estimated for taking and completing the module: 2 semester(s).
This module can be started in the following semesters: Winter and summer semester.
Maximum number of participants
This module is not limited to a number of students.
Registration procedures
The number of participants is limited only in the Laboratory courses, because of lab space.
Registration for the lab courses and integrated course is done online.
Information can be found at:
www.commit.tu-berlin.de,
and it will be given anyway on the first date of the lecture in the required courses.
Registration for the lab courses and integrated course is done online.
Information can be found at:
www.commit.tu-berlin.de,
and it will be given anyway on the first date of the lecture in the required courses.
Recommended reading, lecture notes
Lecture notes
unavailable
Electronic lecture notes
available
Recommended literature
No recommended literature given
Assigned programs
This module is not used in any program.
Miscellaneous
Course material for the various courses will be provided on-line.
Textbooks are indicated as ``non-mandatory desirable reading'', but the courses are self-contained, i.e.,
purchase of textbooks is not mandatory.
Textbooks are indicated as ``non-mandatory desirable reading'', but the courses are self-contained, i.e.,
purchase of textbooks is not mandatory.