
Book contents
- Frontmatter
- Contents
- Preface
- 1 Introduction to information theory
- 2 Finite-state sources
- 3 Channels and linear codes
- 4 Reed–Solomon codes and their decoding
- 5 Source coding
- 6 Information in two-dimensional media
- 7 Constrained two-dimensional fields for storage
- 8 Reed–Solomon codes in applications
- Appendix A Fast arithmetic coding
- Appendix B Maximizing entropy
- Appendix C Decoding of Reed–Solomon code in F (16)
- Index
1 - Introduction to information theory
Published online by Cambridge University Press: 05 June 2012
- Frontmatter
- Contents
- Preface
- 1 Introduction to information theory
- 2 Finite-state sources
- 3 Channels and linear codes
- 4 Reed–Solomon codes and their decoding
- 5 Source coding
- 6 Information in two-dimensional media
- 7 Constrained two-dimensional fields for storage
- 8 Reed–Solomon codes in applications
- Appendix A Fast arithmetic coding
- Appendix B Maximizing entropy
- Appendix C Decoding of Reed–Solomon code in F (16)
- Index
Summary
Introduction
In this chapter we present some of the basic concepts of information theory. The situations we have in mind involve the exchange of information through transmission and storage media designed for that purpose. The information is represented in digital formats using symbols and alphabets taken in a very broad sense. The deliberate choice of the way information is represented, often referred to as coding, is an essential aspect of the theory, and for many results it will be assumed that the user is free to choose a suitable code.
We present the classical results of information theory, which were originally developed as a model of communication as it takes place over a telephone circuit or a similar connection. However, we shall pay particular attention to two-dimensional (2-D) applications, and many examples are chosen from these areas. A huge amount of information is exchanged in formats that are essentially 2-D, namely web-pages, graphic material, etc. Such forms of communication typically have an extremely complex structure. The term media is often used to indicate the structure of the message as well as the surrounding organization. Information theory is relevant for understanding the possibilities and limitation of many aspects of 2-D media and one should not expect to be able to model and analyze all aspects within a single approach.
Entropy of discrete sources
A discrete information source is a device or process that outputs symbols at discrete instances from some finite alphabet A = {x1, x2, …, xr}.
- Type
- Chapter
- Information
- Two-Dimensional Information Theory and CodingWith Applications to Graphics Data and High-Density Storage Media, pp. 1 - 16Publisher: Cambridge University PressPrint publication year: 2009