Shannon defined information as symbols that are uncertain to the receiver. Symbols that are certain may be left out of a message without altering the message's intelligibility. This principle was already in practice by telegraph operators, who would habitually leave out words such as "a" or "the" in order to transmit messages faster. From Shannon's theory, this idea can be carried further with sentences reading something like:

only infrmatn esentil to understandn mst b tranmitd

The amount of information in a message can be quantified by its entropy, which is a measurement of the degree of disorder present within a system. Disorder corresponds to uncertain elements in a message, while order corresponds to the certain elements that may be removed for faster transmission.

An Overview of Information Theory from by Lucent Technologies explains entropy as follows:

Suppose we are watching cars going past on a highway. For simplicity, suppose 50% of the cars are white, 25% are red, 12.5% are yellow, and 12.5% are blue.

Consider the flow of cars as an information source with four colors. A simple way of encoding this source into binary symbols would be to associate each color with two bits -- an average of 2.00 bits per color:

50% white 00
25% red 01
12.5% yellow 10
12.5% blue 11

However, by properly using Information Theory, a better encoding can be constructed:

50% white 0 (0.50 white x 1 bit = .500)
25% red 10 (0.25 red x 2 bits = .500)
12.5% yellow 110 (0.125 yellow x 3 bits = .375)
12.5% blue 111 (0.125 blue x 3 bits = .375)

How is this encoding better? With this code, the average number of bits per car will be less: 1.75 bits per color:

(.500 + .500 + .375 + .375) = 1.750

If a sequence of cars is three white cars, then a red car, then a yellow car, then a blue car, the sequence can be coded unambiguously as:
00010110111 (colors added for clarity)

Information Theory tells us that the entropy of this information source is 1.750 bits per color and thus no encoding scheme will do better than the scheme just described.

For information to be useful, it must be transmitted to a person or a computer. The receiver has to be able to decode the symbols by matching them against what it/she/he already knows. Three steps are involved:

• appropriate symbols are chosen by the sender and properly encoded (e.g., the letters H-E-L-L-O, or H-L-O)
• the message is transmitted through something: sound, broadcast waves, cable, etc.
• the message must be received and tested against the receiver's prior knowledge ("H-L-O -- that means hello, so someone is greeting me")

Thus the rate of information transfer must be taken into account. Factors include:

• the number of symbols that can be transmitted per second
• the amount of noise in the transmission medium, which gives a probability of the frequency of errors in the transmission (e.g., 10% of the time, a 0 will be changed to a 1, or vice versa)
• extra time (and symbols) needed to transmit appropriate error correction information
• extra time (and symbols) needed to transmit redundant information to reduce the probability of error.

Shannon's equations describe the necessary transmission rate, error rate, and channel capacity necessary to send information from one place to another. Also factored in is the percentage of reduction that results from encoding schemes. His work remains the basis for advances in communications technology, which is nowhere near the speed it needs to be to reach Shannon's theoretical limits. When you listen to radio broadcasts over the Internet, go to a live teleconference, or see a live TV newsfeed, you can thank Shannon for making it all possible.