书好效果也很好， 1 Introduction and Preview 1 1.1 Preview of the Book 5 2 Entropy, Relative Entropy, and Mutual Information 2.1 Entropy 13 2.2 Joint Entropy and Conditional Entropy 16 2.3 Relative Entropy and Mutual Information 19 2.4 Relationship Between Entropy and Mutual Information 20 2.5 Chain Rules for Entropy, Relative Entropy, and Mutual Information 22 2.6 Jensen’s Inequality and Its Consequences 25 2.7 Log Sum Inequality and Its Applications 30 2.8 Data-Processing Inequality 34 2.9 Sufficient Statistics 35 2.10 Fano’s Inequality 37 Summary 41 Problems 43 Historical Notes 54

锚定的CorEx：具有最少领域知识的分层主题建模 肺心病的关系防爆夷（COREX）是一个主题模式，产生丰富的主题，最大程度地提供信息有关的一套文件。 与其他主题模型相比，使用CorEx的优势在于，可以根据用户的需要轻松地将其作为无监督，半监督或分层的主题模型来运行。 对于半监督，CorEx允许用户通过“锚词”集成他们的领域知识。 这种集成是灵活的，并允许用户在这些单词的方向上指导主题模型。 这允许采用创新策略来促进主题表示，可分离性和方面。 更一般而言，CorEx的这种实现方式适合于聚类任何稀疏的二进制数据。 如果使用此代码，请引用以下内容： Gallagher，RJ，Reing，K.，Kale，D。和Ver Steeg，G。“。” 计算语言学协会（TACL）的交易，2017年。 入门 安装 可以通过pip安装CorEx主题模型的Python代码： pip install corex

信息论相关基础知识（自然语言处理 NLP）

network information theory（网络信息论）第2章课后题答案

Gallager的Information theory and reliable communication是1976版的，至今仍然是MIT的研究生必读教材。

This comprehensive treatment of network information theory and its applications provides the first unified coverage of both classical and recent results. With an approach that balances the introduction of new models and new coding techniques, readers are guided through Shannon’s point-to-point information theory, single-hop networks, multihop networks, and extensions to distributed computing, secrecy, wireless communication, and networking. Elementary mathematical tools and techniques are used throughout, requiring only basic knowledge of probability, whilst unified proofs of coding theorems are based on a few simple lemmas, making the text accessible to newcomers. Key topics covered include successive cancellation and superposition coding, MIMO wireless communication, network coding, and cooperative relaying. Also covered are feedback and interactive communication, capacity approximations and scaling laws, and asynchronous and random access channels. This book is ideal for use in the classroom, for self-study, and as a reference for researchers and engineers in industry and academia.

信息论基础，第二版，英文版本

Elements of Information Theory (Second Edition) (Thomas, 2006).pdf 做信息论的必读推荐书籍，相当经典

包括香农三大定理，信息论的发展，编码等等

Elements Of Information Theory Elements Of Information Theory