Courses with significant overlap with this course:

Semester of last offering:

Date of approval: dd-mmm-yyyy

Prerequisites:  

Course Contents

Introduction: Entropy, Relative Entropy, Mutual Information Inequalities, Entropy rate. Asymptotic Equipartition Property (AEP): Consequences of the AEP, Typical Sequences, Shannon McMillan Breiman Theorem. Data Compression: Block to variable length codes, Shannon Fano code, Huffman code, variable to fixed length coding Tunstal code, variable to variable length codes/ arithmetic code. Channel capacity: Discrete Memory less Channel, Joint Typicality, Channel Coding Theorem and its converse, Feedback capacity, Source Channel Separation Theorem. Differential Entropy: Definition, Properties. Gaussian Channel: Definition, Parallel Gaussian Channels, Channels with Colored Gaussian Noise, Gaussian Channels with Feedback. Rate Distortion Theory: Rate Distortion Function, Rate Distortion theorem and its converse, Blahut Arimoto Algorithm. Universal Source Coding: Universal codes, Lempel-Ziv codes; LZ 78, LZW, Sliding Window Lempel Ziv algorithm (LZ77). Network Information Theory: Gaussian Multi User Channels, Multiple Access Channel, Broadest Channel, Encoding of Correlated Sources. 

Topics