# Advanced topics in information theory

(Difference between revisions)
 Revision as of 19:54, 4 July 2009 (view source)← Previous diff Revision as of 07:41, 5 July 2009 (view source) (→Reading Group: Advanced Topics in Information Theory)Next diff → Line 2: Line 2: '''Calendar:''' Summer 2009 '''Calendar:''' Summer 2009 + '''Venue:''' LUMS School of Science & Engineering '''Venue:''' LUMS School of Science & Engineering - '''Organizer:''' Abubakr Muhammad - This group meets every week at LUMS to discuss some advanced topics in information theory. + '''Organizer:''' Abubakr Muhammad + This group meets every week at LUMS to discuss some advanced topics in information theory. == Participants == == Participants ==

## Contents

Calendar: Summer 2009

Venue: LUMS School of Science & Engineering

This group meets every week at LUMS to discuss some advanced topics in information theory.

## Participants

• Mubasher Beg
• Shahida Jabeem
• Qasim Maqbool
• Hassan Mohy-ud-Din
• Zartash Uzmi
• Shahab Baqai

## Topics

• Rate distortion theory
• Network information theory
• Kolmogorov complexity
• Quantum information theory

## Sessions

### July 7: Organization. Recap of CS-683

• Basic organization, presentation assignments.
• Review of Information theory ideas
• Entropy, AEP, Compression and Capacity

Entropy of a random variable is given by $H(X) = -\sum_{x \in \mathcal{X}} p(x) \log p(x).$

The capacity of a channel is defined by $\mathcal{C} = \max_{p(x)} I(X; Y).$

Compression and Capacity determine the two fundamental information theoretic limits of data transmission, $H \leq R \leq \mathcal{C}.$

• A review of Gaussain channels and their capacities.
• Let us take these analysis one step further. How much do you loose when you cross these barriers?
• We saw one situation when you try to transmit over the capacity. By Fano's inequality $P_e \geq 1 - \frac{C}{R} - \frac{1}{nR}$

• Rate distortion: A theory for lossy data compression.