Advanced topics in information theory


Revision as of 19:54, 4 July 2009 by Abubakr (Talk | contribs)
Jump to: navigation, search


Reading Group: Advanced Topics in Information Theory

Calendar: Summer 2009 Venue: LUMS School of Science & Engineering Organizer: Abubakr Muhammad

This group meets every week at LUMS to discuss some advanced topics in information theory.


  • Mubasher Beg
  • Shahida Jabeem
  • Qasim Maqbool
  • Muhammad Bilal
  • Muzammad Baig
  • Hassan Mohy-ud-Din
  • Zartash Uzmi
  • Shahab Baqai
  • Abubakr Muhammad


  • Rate distortion theory
  • Network information theory
  • Kolmogorov complexity
  • Quantum information theory


July 7: Organization. Recap of CS-683

  • Basic organization, presentation assignments.
  • Review of Information theory ideas
  • Entropy, AEP, Compression and Capacity

Entropy of a random variable is given by

H(X) = -\sum_{x \in \mathcal{X}} p(x) \log p(x).

The capacity of a channel is defined by

\mathcal{C} = \max_{p(x)} I(X; Y).

Compression and Capacity determine the two fundamental information theoretic limits of data transmission, H \leq R \leq \mathcal{C}.

  • A review of Gaussain channels and their capacities.
  • Let us take these analysis one step further. How much do you loose when you cross these barriers?
  • We saw one situation when you try to transmit over the capacity. By Fano's inequality

P_e \geq 1 - \frac{C}{R} - \frac{1}{nR}

  • Rate distortion: A theory for lossy data compression.

July 16: Rate distortion theory - I

July 21: Rate distortion theory - II

July 28: Network Information theory- I

Aug 04: Network Information theory- II

Aug 11: Wireless networks, cognitive radios

Aug 18: Multiple access channels, network coding techniques

Personal tools