An Introduction to Single-User Information Theory

This second edition is extensively updated to reflect the growing synergy between information theory and artificial intelligence (AI) . Key enhancements include: New Content Focused on AI & Machine Learning: A major new chapter introduces generalized information and statistical measures (such as parameterized Rényi, Jensen, and f-divergence), alongside expanded coverage of the information bottleneck principle , cross-entropies , and generalized information inequalities , all connected to modern ML applications like deep learning and generative models. Comprehensive Updates Throughout: All existing chapters have been revised and expanded , the appendix enhanced , and chapter problem sets significantly enlarged based on the first edition. The book presents a succinct and mathematically rigorous treatment of the main pillars of Shannon’s information theory, discussing the fundamental concepts and indispensable results of Shannon’s mathematical theory of communications. It includes six meticulously written core chapters (with accompanying problems), emphasizing the key topics of information measures (classical and generalized); lossless and lossy data compression; channel coding; and joint source-channel coding for single-user (point-to-point) communications systems. It also features two appendices covering necessary background material in real analysis and in probability theory and stochastic processes. Improved Accuracy & Resources: Errors from the first edition have been corrected, and the bibliography and index are thoroughly updated . A separate, 400-page instructor’s solutions manual is also available. The book is ideal for a one-semester foundational course on information theory for senior undergraduate and entry-level graduate students in mathematics, statistics, engineering, and computing and information sciences.