EE4253 Digital Communications Department of Electrical and Computer Engineering - University of New Brunswick, Fredericton, NB, Canada

Error Correction and the Hamming Code

The use of simple parity allows detection of single bit errors in a received message. Correction of such errors requires more information, since the position of the bad bit must be identified if it is to be corrected. (If a bad bit can be found, then it can be corrected by simply complementing its value.) Correction is not possible with one parity bit since any bit error in any position produces exactly the same information - "bad parity".

If more bits are included with a message, and if those bits can be arranged such that different errored bits produce different error results, then bad bits could be identified. In a 7-bit message, there are seven possible single bit errors, so three error control bits could potentially specify not only that an error occured but also which bit caused the error.

Similarly, if a family of codewords is chosen such that the minimum distance between valid codewords is at least 3, then single bit error correction is possible. This distance approach is "geometric", while the above error-bit argument is 'algebraic'.

Either of the above arguments serves to introduce the Hamming Code, an error control method allowing correction of single bit errors.

The Hamming Code

Consider a message having four data bits (D) which is to be transmitted as a 7-bit codeword by adding three error control bits. This would be called a (7,4) code. The three bits to be added are three EVEN Parity bits (P), where the parity of each is computed on different subsets of the message bits as shown below.

 7 6 5 4 3 2 1 D D D P D P P 7-BIT CODEWORD D - D - D - P (EVEN PARITY) D D - - D P - (EVEN PARITY) D D D P - - - (EVEN PARITY)

 Why Those Bits? - The three parity bits (1,2,4) are related to the data bits (3,5,6,7) as shown at right. In this diagram, each overlapping circle corresponds to one parity bit and defines the four bits contributing to that parity computation. For example, data bit 3 contributes to parity bits 1 and 2. Each circle (parity bit) encompasses a total of four bits, and each circle must have EVEN parity. Given four data bits, the three parity bits can easily be chosen to ensure this condition. It can be observed that changing any one bit numbered 1..7 uniquely affects the three parity bits. Changing bit 7 affects all three parity bits, while an error in bit 6 affects only parity bits 2 and 4, and an error in a parity bit affects only that bit. The location of any single bit error is determined directly upon checking the three parity circles.

For example, the message 1101 would be sent as 1100110, since:

 7 6 5 4 3 2 1 1 1 0 0 1 1 0 7-BIT CODEWORD 1 - 0 - 1 - 0 (EVEN PARITY) 1 1 - - 1 1 - (EVEN PARITY) 1 1 0 0 - - - (EVEN PARITY)

When these seven bits are entered into the parity circles, it can be confirmed that the choice of these three parity bits ensures that the parity within each circle is EVEN, as shown here.

It may now be observed that if an error occurs in any of the seven bits, that error will affect different combinations of the three parity bits depending on the bit position.

For example, suppose the above message 1100110 is sent and a single bit error occurs such that the codeword 1110110 is received:

```      transmitted message                           received message
1 1 0 0 1 1 0           ------------>          1 1 1 0 1 1 0
BIT:  7 6 5 4 3 2 1                            BIT:  7 6 5 4 3 2 1
```

The above error (in bit 5) can be corrected by examining which of the three parity bits was affected by the bad bit:

 7 6 5 4 3 2 1 1 1 1 0 1 1 0 7-BIT CODEWORD 1 - 1 - 1 - 0 (EVEN PARITY) NOT! 1 1 1 - - 1 1 - (EVEN PARITY) OK! 0 1 1 1 0 - - - (EVEN PARITY) NOT! 1

In fact, the bad parity bits labelled 101 point directly to the bad bit since 101 binary equals 5. Examination of the 'parity circles' confirms that any single bit error could be corrected in this way.

The value of the Hamming code can be summarized:

1. Detection of 2 bit errors (assuming no correction is attempted);
2. Correction of single bit errors;
3. Cost of 3 bits added to a 4-bit message.

The ability to correct single bit errors comes at a cost which is less than sending the entire message twice. (Recall that simply sending a message twice accomplishes no error correction.)

Hamming Distance = 3

The Hamming Code allows error correction because the minimum distance between any two valid codewords is 3. In the figure below, two valid codewords in 8 possible 3-bit codewords are arranged to have a distance of 3 between them. It takes 3 bit changes (errors) to move from one valid codeword 000 to the other 111. If the codeword 000 is transmitted and a single bit error occurs, the received word must be one of {001,010,100}, any of which is easily identified as an invalid codeword, and which could only have been 000 before transmission.

Sixteen Valid Codewords

 7 6 5 4 3 2 1 The Hamming Code code essentially defines 16 valid codewords within all 128 possible 7-bit codewords. The sixteen words are arranged such that the minimum distance between any two words is 3. These words are shown in this table, where it is left as an exercise to check that from any codeword N={0..F} in the table to any other word M, the distance is at least 3. Example: For N=3, codeword 3 = 0011110 is expected to be a distance of at least 3 from all the other codewords. The distance is 4 between 3 = 0011110 and 0 = 0000000 The distance is 3 between 3 = 0011110 and 1 = 0000111 The distance is 3 between 3 = 0011110 and 2 = 0011001 ... The distance is 4 between 3 = 0011110 and D = 1100110 The distance is 4 between 3 = 0011110 and E = 1111000 The distance is 3 between 3 = 0011110 and F = 1111111 Therefore, codeword 3 is a distance of at least 3 from any other valid codeword. 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 2 0 0 1 1 0 0 1 3 0 0 1 1 1 1 0 4 0 1 0 1 0 1 0 5 0 1 0 1 1 0 1 6 0 1 1 0 0 1 1 7 0 1 1 0 1 0 0 8 1 0 0 1 0 1 1 9 1 0 0 1 1 0 0 A 1 0 1 0 0 1 0 B 1 0 1 0 1 0 1 C 1 1 0 0 0 0 1 D 1 1 0 0 1 1 0 E 1 1 1 1 0 0 0 F 1 1 1 1 1 1 1

 The Distance Argument Looking again at the Venn diagram (at right) it can be observed that a change in any of the data bits (3,5,6,7) necessary changes at least two other bits in the codeword. For example, given a valid Hamming codeword, a change in bit 3 changes three bits (1,2,3) such that the new codeword is a distance (d=3) from the initial word. The clever arrangement of the Hamming codewords ensures that this is the case for every valid codeword in the set.

A Final Note

Any set of codewords is useful for error control provided that the minimum distance between any two of them is some value D. (some may be more that D but none will be less than D)

So there is no unique set of codewords with L=7 and D=3. The Hamming code shown here (L=7,D=3) is useful because it is easy to generate and to check this particular set of codewords by hand. The distances would still be the same if you swapped two columns or complemented the bits in any column, but the codewords would look very different (and the Venn diagrams would not work!).

To explore this subject further, continue with Hamming Code Revisited

To see sets of codewords with various distance properties, Online Hamming Code Tool

The value of carefully choosing error control schemes is self-evident in the Hamming Code. Still, for very long messages another approach is desirable.

 Mon May 20 16:28:11 ADT 2013 Last Updated: 19 SEP 2005 Richard Tervo [ tervo@unb.ca ] Back to the course homepage...