JTG/IEEE ITSoc Summer School in Information Theory, Signal Processing, Telecommunication, and Networking @ IIT Kanpur

Abstract

Claude Shannon's 1948 "A Mathematical Theory of Communication" provided the basis for the digital communication revolution. As part of that ground-breaking work, he identified the greatest rate (capacity) at which data can be communicated over a noisy channel. His proposed algorithm used on random codes and a code centric maximum Maximum Likelihood (ML) decoding, where channel outputs are compared to all possible input codewords to select the most likely candidate based on the observed channel output. Despite its mathematical elegance, his code centric decoding algorithm is impractical from a complexity perspective and much work in the intervening 70 years has focused on co-designing codes and decoders that enable reliable communication at high rates. In collaboration with Ken Duffy and his group, we introduce a new algorithm, Guessing Random Additive Noise Deceasing (GRAND) for a noise-centric, rather than code-centric, ML decoding. The receiver rank orders noise effect sequences from most likely to least likely, and guesses accordingly. When inverting, in decreasing order of likelihood, noise effect sequences from the received signal, the first instance that results in an element of the code-book is the ML decoding. Our results show that, with GRAND, even extremely simple codes, such as CRCs, match or outperform state of the art code/decoder pairs, indicating that the choice of decoder is likely to be more important than that of code.

Topics:

• Brief introduction to entropy
• The channel coding theorem via GRAND
• Error exponents and success
• GRAND as an algorithm
• Soft information in GRAND
• Decoding with GRAND in practice - algorithms and hardware implementations.

Speaker Details

2021 Padovani Lecturer Muriel Médard

Muriel Médard is the Cecil H. Green Professor in the Electrical Engineering and Computer Science (EECS) Department at MIT. She leads the Network Coding and Reliable Communications Group at the Research Laboratory for Electronics at MIT. Her research interests include network coding, information theory, wireless networks, and optical networking.

Abstract

In this short course, the instructor will give a detailed introduction to the state-of-the-art in molecular communication. He will start with a theoretical perspective, showing how molecular communication fits with the standard framework for analyzing communication systems. He will give various models for molecular communication, good communication strategies, and information-theoretic analysis. He will also give an introduction to the necessary concepts of chemistry and biology, at a level appropriate for electrical engineers. He will then move on to a practical perspective, showing how to validate these results experimentally, and describing two successful, low-cost, tabletop experimental systems that have already been used in published experiments. Finally, he will give an overview of the many open problems in this exciting and growing field of research.

Topics:

• Basic introduction to molecular communication systems
• Diffusion in molecular communication
• Modulation and detection schemes for molecular communication
• he additive inverse Gaussian noise model and related models
• Molecular MIMO
• Information-theoretic aspects of molecular communication
• The basics of biological communication
• Biological molecular communication: Signal transduction
• Mass action kinetics and mathematical models of signal transduction
• Information-theoretic aspects of signal transduction
• Experimental approaches to molecular communication

Speaker Details

Andrew Eckford

Andrew Eckford is an Associate Professor in the Department of Electrical Engineering and Computer Science at York University, Toronto, Ontario. His research interests include the application of information theory to biology, and the design of communication systems using molecular and biological techniques. His research has been covered in media including The Economist, The Wall Street Journal, and IEEE Spectrum. His research received the 2015 IET Communications Innovation Award, and was a finalist for the 2014 Bell Labs Prize. He is also a co-author of the textbook Molecular Communication, published by Cambridge University Press. Andrew received the B.Eng. degree from the Royal Military College of Canada in 1996, and the M.A.Sc. and Ph.D. degrees from the University of Toronto in 1999 and 2004, respectively, all in Electrical Engineering. Andrew held postdoctoral fellowships at the University of Notre Dame and the University of Toronto, prior to taking up a faculty position at York in 2006. He has held courtesy appointments at the University of Toronto and Case Western Reserve University. In 2018, he was named a Senior Fellow of Massey College, Toronto.

Abstract:

Communication system design has traditionally relied on developing a mathematical model and producing optimized algorithms for that model. However, with the increasing access to data and computing resources, a complementary data-driven approach based on machine learning has gained interest in recent years. This short course provides a brief introduction to machine learning that is tailored for communication and information theory researchers. The first module will provide an overview of statistical learning that will lead into the discussion of the types of communication system design problems that can benefit from machine learning. A case study exploring the connection of machine learning to point processes in the context of subset selection problems in wireless networks will also be presented. The second module will focus on statistical estimation. Popular supervised learning algorithms will be interpreted as ML and MAP estimators of appropriate underlying statistical models. The last two modules will focus on unsupervised learning, including discussions on k-means, expectation maximization, as well as detailed case studies related to distributed learning in wireless networks and codebook design in MIMO systems.

Topics:

• Introduction to Statistical Learning
• Role of Machine Learning in Communications
• Determinantal Learning for Subset Selection in Wireless Networks
• Statistical Estimation
• Supervised Learning: Introduction, Interpretation as ML/MAP Estimators
• Unsupervised Learning: Introduction, k-means, and Expectation Maximization
• k-means Clustering on a Grassmann Manifold for MIMO Codebook Design
• Distributed Learning in Wireless Networks

Speaker Details

Harpreet S. Dhillon

Harpreet S. Dhillon received the B.Tech. degree in electronics and communication engineering from IIT Guwahati in 2008, the M.S. degree in electrical engineering from Virginia Tech in 2010, and the Ph.D. degree in electrical engineering from the University of Texas at Austin in 2013. After serving as a Viterbi Postdoctoral Fellow at the University of Southern California for a year, he joined Virginia Tech in 2014, where he is currently an Associate Professor of electrical and computer engineering and the Elizabeth and James E. Turner Jr. ¿56 Faculty Fellow. He is a Clarivate Analytics Highly Cited Researcher and has coauthored five best paper award recipients including the 2014 IEEE Leonard G. Abraham Prize, the 2015 IEEE ComSoc Young Author Best Paper Award, and the 2016 IEEE Heinrich Hertz Award. In 2020, he received Early Achievement Awards from the IEEE Communication Theory Technical Committee (CTTC) and the IEEE Radio Communications Committee (RCC). He was named the 2017 Outstanding New Assistant Professor, the 2018 Steven O. Lane Junior Faculty Fellow, the 2018 College of Engineering Faculty Fellow, and the recipient of the 2020 Dean's Award for Excellence in Research by Virginia Tech. His other academic honors include the 2008 Agilent Engineering and Technology Award, the UT Austin MCD Fellowship, the 2013 UT Austin WNCG leadership award, and the inaugural IIT Guwahati Young Alumni Achiever Award 2020. He is a senior member of IEEE and serves on the editorial boards of three of its journals.

Ashutosh Deepak Gore

Ashutosh Deepak Gore obtained his B.Tech. and Ph.D. degrees from IIT Bombay, and M.S. degree from University of Hawaii, all in Electrical Engineering. He worked in Nortel Networks, USA from 2000 to 2002. From 2008 onwards, he has worked in various semiconductor companies in India, viz. Marvell Semiconductor, Broadcom, Samsung R&D and Qualcomm, where he is currently Principal Engineer/Manager. His work in Qualcomm focusses on PHYsical layer algorithms and functional reference models for next generation (IEEE 802.11be compliant) wireless local area networks. He joined Qualcomm in May 2016 and seeded the WLAN PHY modeling group in Qualcomm Bangalore. His team developed several key PHY blocks in multiple generations of WLAN AP & STA chipsets. He has 10 US granted patents and 14 approved for filing. He has published 3 IEEE journal papers and 8 conference papers.

Title: 802.11be WLAN Physical Layer

This presentation gives an introduction to IEEE 802.11be - the next generation wireless LAN standard. It first covers the evolution of WLAN for 3 decades, and then gets into the top 7 features of WiFi7. It describes the EHT physical layer packet structure and touches upon advanced features in EHT OFDMA such as preamble puncturing and embedded channel hierarchy.

Amitalok Budkuley

Amitalok J. Budkuley is an assistant professor in the Dept. of Electronics and Electrical Communication Engineering at the Indian Institute of Technology Kharagpur since 2019. He received his B. Engg. Degree in Electronics and Telecommunications Engineering from Goa University, in 2007, and his M. Tech. and Ph. D. degree in Electrical Engineering from the Indian Institute of Technology Bombay, Mumbai, India in 2009 and 2017, respectively. In between, he spent some time in the industry working with Cisco Systems Inc. From 2016 to 2019, he was at the Dept. of Information Engineering, The Chinese University of Hong Kong (CUHK) as a research assistant and then as a post-doctoral fellow. His research interests include information theory, wireless communications, signal processing for communication systems, and game theory.

Title: Unconditionally Secure Commitment over Constrained Noisy Channels

Imagine playing a game of rock-paper-scissors, albeit in this time of social distancing. A fundamental conundrum is the following: how does one simulate and verify an instance of simultaneous play, an intrinsic feature of this game, among two parties who are fundamentally distrustful and not collocated? Commitment is a powerful cryptographic primitive which helps realize the above functionality. In this talk, we leverage constrained noisy channels for realizing unconditionally secure or information-theoretically secure commitment protocols. We characterize the commitment capacity under general channel constraints and present an interesting dual view of our capacity characterization. Joint work with Manideep Mamindlapally (IIT Kharagpur), Anuj Kumar Yadav (IIT Patna), and Manoj Mishra (NISER, HBNI, Bhubaneshwar)

Shashank Vatedka

Shashank Vatedka received his MSc (Engg) and Ph.D. degrees from the Department of Electrical Communication Engineering, Indian Institute of Science. He subsequently did postdocs in the Institute of Network Coding (Chinese University of Hong Kong) and Telecom Paris (France) before joining the Department of Electrical Engineering, IIT Hyderabad, where he is currently an assistant professor. He was a recipient of the TCS research scholarship during his doctoral studies, the Seshagiri Kaikini medal for best Ph.D. thesis in the Dept of ECE at IISc, runner-up for the best paper award at SPCOM 2020, and the best poster award at the 2021 Stanford Compression Workshop. His research interests are in information theory and coding, with applications to security, data compression, and statistical inference.

Title: Data compression with locality

In this talk, I will present some of our recent results on data compression algorithms that simultaneously allow local decoding and local update. In applications such as cloud storage and storage of genomic collections, we are often interested in making frequent accesses and modifications to short fragments of the raw data directly in its compressed form. Traditional compressors such as Lempel-Ziv are inefficient in this regard, as even accessing/replacing a single bit of the raw file requires us to decompress/recompress the entire sequence. I will talk about entropy-achieving compressors that attempt to minimize the local decodability, which is the number of compressed bits that need to be read to recover a single bit of the raw file, and the update efficiency, which is the number of compressed bits that need to be read and written to update a single bit of the raw file. This is joint work with Aslan Tchamkerten (Telecom Paris) and Venkat Chandar (DE Shaw, NY)

Avhishek Chatterjee

Avhishek Chatterjee joined the Department of Electrical Engineering of the Indian Institute of Technology Madras as an assistant professor in October 2017. Before that, he was a postdoctoral research associate at the Coordinated Science Laboratory of the University of Illinois at Urbana-Champaign. He obtained Ph.D. in Electrical and Computer Engineering in 2015 from The University of Texas at Austin, a Master of Engineering in Telecommunication from the Electrical Communication Engineering department of Indian Institute of Science, Bangalore in 2008, and a Bachelor of Engineering in Electronics and Telecommunication Engineering from Jadavpur University, Kolkata (India) in 2006. He worked as a member of technical staff at Alcatel-Lucent Bell Labs, India, from 2008 to 2011. His research interest lies in theoretical studies of dynamics, optimal designs, and operations of stochastic networks.

Title: Decoherence in the Buffer and the Classical Capacity of Quantum Channels

Channel noise is inherent to both classical and quantum communication systems. However, in practice, quantum communication systems encounter an additional source of noise: decoherence of quantum bits (qubits) while waiting in a memory (buffer) for transmission. In this talk, we propose a quantum channel model that captures this phenomenon and then study its classical capacity. This channel model also has applications in quantum computing systems that suffer from a similar kind of decoherence. The capacity results offer insights into the design of quantum information processing systems. (No pAvhishekrior knowledge in quantum communication or computation is required to follow this talk.) Joint work with Prabha Mandayam (IIT Madras), Krishna Jagannathan (IIT Madras), Vikesh Siddhu (CU Boulder), and Sridhar Taiyur (CMU).

Piyush Srivastava

Piyush Srivastava is a Reader at the School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai. His research is broadly in applications of probabilistic ideas in computer science. Before coming to TIFR, he studied at IIT Kanpur and UC Berkeley and was a post-doctoral researcher at Caltech.

Title: An invitation to causal inference

This talk will be a short survey of Pearl's influential formalization of causal inference in terms of probabilistic graphical models and the connections of this framework with information theory, focusing on questions of robustness. Parts of this talk are based on joint work with Spencer Gordon, Vinayak Kumar, and Leonard Schulman.

Varun Narayanan

I am currently awaiting to defend my PhD thesis which I completed at TIFR, where Prof. Vinod Prabhakaran was my advisor. I am currently awaiting the defense of my thesis. I am currently a postdoctoral researcher at Technion, Israel hosted by Prof. Yuval Ishai and Eyal Kushilevitz. My areas of interest include secure multiparty computation, applications of information theory in security and privacy, and cryptography in general.

Title: Computationally Secure Computation from One-Way Noisy Communication

Can a sender encode a pair of messages (m0, m1) jointly, and send their encoding over (say) a binary erasure channel, so that the receiver can decode exactly one of the two messages and the sender does not know which one? Garg et al. (Crypto 2015) showed that this is information-theoretically impossible. We show how to circumvent this impossibility by assuming that the receiver is computationally bounded, settling for an inverse- polynomial security error (which is provably necessary), and relying on ideal obfuscation. Our solution creates a "computational anti-correlation" between the events of receiving m0 and receiving m1 by exploiting the anti-concentration of the binomial distribution. The ideal obfuscation primitive in our construction can either be directly realized using (stateless) tamper-proof hardware, yielding an unconditional result. As a corollary, we get similar feasibility results for general secure computation of sender-receiver functionalities by leveraging the completeness of the above random oblivious transfer functionality.

Kota Srinivas Reddy

Kota Srinivas Reddy recently gave his Ph.D. defense in Electrical Engineering with IIT Bombay, India. His Ph.D. advisor was Prof. Nikhil Karamchandani. He received the M.Tech. Degree in Telecommunication Systems Engineering from IIT Kharagpur, in 2014. His research interests are in the areas of probability theory, information theory, coded caching, and learning theory.

Title: Structured index coding problem and multi-access coded caching

Index coding and coded caching are two active research topics in information theory with strong ties to each other. Motivated by the multi-access coded caching problem, we study a new class of structured index coding problems (ICPs) formed by the union of several symmetric ICPs. We derive upper and lower bounds on the optimal server transmission rate for this class of ICPs and demonstrate that they differ by at most a factor of two. Finally, we apply these results to the multi-access coded caching problem to derive better bounds than state-of-the-art.

Deekshith P K

Deekshith P K obtained his B.Tech degree in Electronics and Communication Engineering from Government Engineering College, Thrissur, Kerala. His doctoral thesis, done under the guidance of Prof. Vinod Sharma, Dept. of ECE, IISc, Bangalore, concerns an information-theoretic analysis of point-to-point and multi-user wireless channels with energy harvesting and finite blocklength constraints. He is currently with the Mobile Battery Division, part of Samsung Advanced Institute of Technology (SAIT), Samsung R&D Institute-Bangalore.

Title:Finite Blocklength Rates over a Fading Channel with an Energy Harvesting Transmitter

In this talk, we will present a finite blocklength analysis for a block fading channel with an energy harvesting transmitter. We characterize tractable lower and upper bounds for the second-order coefficient in the approximation of maximal coding rate at a given codeword length and the average probability of error. The bounds are derived under the assumption of full causal channel state information (CSI) at the transmitter, referred to as CSIT, and full CSI at the receiver (CSIR). As a special case, we recover the corresponding bounds for no CSIT and full CSIR as well.

Mohammad Ishtiyaq Qureshi

Mohammad Ishtiyaq Qureshi received his B. Eng. in Electronics and Communication Engineering from Rajiv Gandhi Proudyogiki Vishwavidyalaya, Bhopal, M. P., India in 2012. He received his M. Des. in Communication Systems in 2015 from the Indian Institute of Information Technology, Design, and Manufacturing, Kancheepuram, T. N., India. Since February 2016, he has been a Ph. D. student in the School of Computing and Electrical Engineering at the Indian Institute of Technology Mandi, H. P., India , under the supervision of Dr. Satyajit Thakor. Currently, he is working as an Associate-Intern at IIT Mandi iHub and HCI Foundation. His research interests center around information theory and network coding.

Title:On the Information Flow in Undirected Unicast Networks

One of the important unsolved problems in information theory is the conjecture that network coding has no rate benefit over routing in undirected unicast networks. If the conjecture is true, then the undirected unicast network information capacity is the same as the routing capacity. In this talk, we present an upper bound, called the partition bound, on the symmetric rate for information flow in general undirected unicast networks. We show explicit routing solutions achieving the partition bound for a class of complete n-partite networks called Type-I n-partite networks. Recently, the conjecture was proved for a new class of networks, and it was shown that all the network instances for which the conjecture is proved previously are elements of this class. We show the existence of a network outside of the class of networks with unverified conjecture such that the partition bound is tight and attainable by routing.