Book contents
- Frontmatter
- Contents
- Preface
- Acknowledgements
- List of symbols
- List of abbreviations
- 1 Introduction
- 2 Deterministic signal analysis
- 3 Random signal analysis
- 4 Information theory and channel coding
- 5 Communication link analysis
- 6 Modulation
- 7 Demodulation
- 8 Spread spectrum
- 9 Intersymbol interference and equalization
- 10 Fading channels
- Index
- References
4 - Information theory and channel coding
Published online by Cambridge University Press: 05 June 2012
- Frontmatter
- Contents
- Preface
- Acknowledgements
- List of symbols
- List of abbreviations
- 1 Introduction
- 2 Deterministic signal analysis
- 3 Random signal analysis
- 4 Information theory and channel coding
- 5 Communication link analysis
- 6 Modulation
- 7 Demodulation
- 8 Spread spectrum
- 9 Intersymbol interference and equalization
- 10 Fading channels
- Index
- References
Summary
Introduction
In this chapter we present the applicability of probability theory and random variables to the formulation of information theory pioneered by Claude Shannon in the late 1940s. Information theory introduces the general idea of source coding and channel coding. The purpose of source coding is to minimize the bit rate required to represent the source (represented mathematically by a discrete random variable) with a specified efficiency at the output of the source coder. On the other hand, the goal of channel coding is to maximize the bit rate at the input of the channel encoder so that code words can be transmitted through the channel with a specified reliability. Both source coding and channel coding can be achieved with the knowledge of the statistics of both the source and channel.
Entropy of a discrete source
Information comes from observing the outcome of an event. Common events occur frequently (high probability) and therefore carry little information. On the other hand, rare events occur infrequently (low probability) and hence carry much more information. In 1928 R. V. L. Hartley proposed a logarithmic measure of information that illustrates this observation.
- Type
- Chapter
- Information
- Theory and Design of Digital Communication Systems , pp. 122 - 176Publisher: Cambridge University PressPrint publication year: 2010