Chapter 1 Introduction 1.1 Concept of information 1.2 History of information theory 1.3 Information, messages and signals 1.4 Communication system model 1.5 Information theory applications 1.5.1 Electrical engineering (communication theory) 1.5.2 Computer science (algorithmic complexity) Exercises Chapter 2 Statistical Measure of Information 2.1 Information of random events 2.1.1 Self-information 2.1.2 Conditional self-information 2.1.3 Mutual information of events 2.2 Information of discrete random variables 2.2.1 Entropy of discrete random variables 2.2.2 Joint entropy 2.2.3 Conditional entropy 2.2.4 Mutual information of discrete random variables 2.3 Relationship between entropy and mutual information 2.4 Mutual information and entropy of continuous random variables 2.4.1 Mutual information of continuous random variabies 2.4.2 Entropy oI continuous random variables Exercises Chapter 3 Discrete Source and Its Entropy Rate 3.1 Mathematical model of source 3.1.1 Discrete source and continuous source 3.1.2 Simple discrete source and its extension 3.1.3 Memoryless source and source with memory 3.2 Discrete memoryles source 3.2.1 Definition 3.2.2 Extension of discrete source 3.3 Discrete stationary source 3.3.1 Definition 3.3.2 Entropy rate of discrete stationary source 3.4 Discrete Markov source 3.4.1 Markov chain 3.4.2 Transition probability 3.4.3 Markov source and its entropy rate Exercises Chapter 4 Lossless Source Coding and Data Compression 4.1 Asymptotic equipartition property and typical sequences 4.2 Lossless source coding 4.2.1 Encoder 4.2.2 Blockcode 4.2.3 Fixed length code 4.2.4 Variable length code 4.3 Data compression 4.3.1 Shannon coding 4.3.2 Huffman coding
4.3.3 Fano coding Exercises Chapter 5 Discrete Channel and Its Capacity 5.1 Mathematical model of channel 5.2 Discrete memoryless channel 5.2.1 Mathematical model o{ discrete memoryless channel 5.2.2 Simple DMC 5.2.3 Extension of discrete memoryless channel 5.3 Channel combination 5.4 Channel capacity 5.4.1 Concept of channel capacity 5.4.2 Channel capacity of several special discrete channels 5.4.3 Channel capacity of symmetric channels 5.4.4 Channel capacity of extended DMC 5.4.5 Channel capacity of independent parallel DMC 5.4.6 Channel capacity of the sum channel 5.4.7 Channel capacity of general discrete channels Exercises Chapter 6 Noisy-channel Coding 6.1 Probability of error 6.2 Decoding rules 6.3 Channel coding 6.3.1 Simple repetition code 6.3.2 Linear code 6.4 Noisy-channel coding theorem Exercises Chapter 7 Rate Distortion 7.1 Quantization 7.2 Distortion definition 7.2.1 Distortion function 7.2.2 Mean distortion 7.3 Rate distortion function 7.3.1 Fidelity criterion for given channel 7.3.2 Definition of rate distortion function 7.3.3 Property of rate distortion function 7.4 Rate distortion theorem and the converse 7.5 The ea|culation of rate distortion function Exercises Chapter 8 Continuous Source find Its Entropy Rate 8.1 Continuous source 8.2 Entropy of continuous source 8.3 Maximum entropy of continuous source 8.4 Joint entropy, conditional entropy and mutual information for continuous random variables 8.5 Entropy rate of continuous source 8.6 Rate distortion for continuous source Exercises Chapter 9 Continuous Channel and Its Capacity 9.1 Capacity of continuous channel 9.1.1 Capacity of discrete-time channel 9.1.2 Capacity of continuous time channel
9.2 The Gaussian channel 9.3 Band-limited channels 9.4 Coding theorem for continuous channel Exercises Chapter 10 Maximum Entropy and Spectrum Estimation 10.1 Maximum entropy probability distribution 10.1.1 Maximum entropy distribution 10.1.2 Examples 10.2 Maximum entropy spectrum estimation 10.2.1 Burg's max entropy theorem 10.2.2 Maximum entropy spectrum estimation Exercises Chapter 11 Experiments of Information Theory 11.1 Measure of information 11.1.1 Information calculator 11.1.2 Properties of entropy 11.2 Simulation of Markov source 11.3 Performance simulation for source coding 11.3.1 Shannon coding 11.3.2 Huffman coding 11.3.3 Fano coding 11.4 Simulation of BSC 11.5 Simulation of the cascade channel 11.6 Calculation of channel capacity 11.7 Decoding rules 11.8 Performance demonstration of channel coding References
[