# Advanced topics in information theory

### From CYPHYNETS

# Reading Group: Advanced Topics in Information Theory

Summer 2009

## Participants

- Mubasher Beg
- Shahida Jabeem
- Qasim Maqbool
- Muhammad Bilal
- Muzammad Baig
- Hassan Mohy-ud-Din
- Zartash Uzmi
- Shahab Baqai
- Abubakr Muhammad

## Topics

- Rate distortion theory
- Network information theory
- Kolmogorov complexity
- Quantum information theory

## Sessions

### July 7: Organization. Recap of CS-683

- Basic organization, presentation assignments.

- Review of Information theory ideas

- Entropy, AEP, Compression and Capacity

Entropy of a random variable is given by

The capacity of a channel is defined by

Compression and Capacity determine the two fundamental information theoretic limits of data transmission,

- A review of Gaussain channels and their capacities.

- Let us take these analysis one step further. How much do you loose when you cross these barriers?

- We saw one situation when you try to transmit over the capacity. By Fano's inequality

- Rate distortion: A theory for lossy data compression.