Using Frequency Tables to Crack Substitution Ciphers

Why Counting Letters Works

Every language has habits. In English prose, the letter E turns up roughly once in every eight letters, whether you are reading a shopping list, a novel or a ransom note. T, A, O, I and N follow behind in a fairly predictable order. Substitution ciphers scramble which letter stands for which, but they cannot hide how often each one appears. A ciphertext stuffed with Qs is probably hiding a lot of Es. That single insight is the engine behind most beginner codebreaking, and it costs nothing but a pencil and a little patience.

Building a Frequency Table by Hand

A frequency table is just a tally. Write the alphabet down the side of a sheet of paper, then work through the ciphertext one letter at a time, putting a mark next to each one you meet. Five-bar gates keep it tidy. When you finish, add up each row.

A few practical tips:

• Strip out spaces and punctuation first, or leave them in as extra clues — punctuation often reveals sentence boundaries, which becomes useful later.
• Count in chunks of fifty or a hundred letters and check your running totals against the total length. A miscount early on will quietly poison everything that follows.
• Convert tallies to percentages if you want to compare against known English letter frequencies. If your ciphertext runs to 200 letters and K appears 26 times, that is 13 per cent — E territory.
• Do a second pass counting from the end of the text back to the start. It catches letters you skimmed past the first time.

Reading the Table: What the Numbers Tell You

Rank your letters from most to least common. The top of the list is where the guessing begins.

• E is the most common letter in English, followed by T, A, O, I, N, S, H and R. Your most frequent ciphertext letter is very often E.
• The rarest letters — Z, Q, X and J — usually stay rare in ciphertext too. If a cipher letter appears only once or twice, it probably came from one of these.
• A letter appearing around 2 per cent of the time is a decent candidate for C, U, M, W, F, G, Y or P.
• If a single letter stands alone as a word, it is almost certainly A or I. That is one of the fastest wins available.

Treat all of this as a starting point rather than a rule book. The aim is to cut the possibilities from twenty-six down to two or three, then let the surrounding words decide between them.

Looking Beyond Single Letters

Single-letter counts get you moving; pairs and triples finish the job. Once you have a rough mapping, tally the two-letter combinations. TH, HE, IN, ER, AN and RE are the workhorses of English, so a ciphertext pair that keeps reappearing is very likely one of them.

Other patterns worth hunting for:

• Double letters. LL, EE, SS, OO, TT and FF are common; doubles narrow down words sharply, especially at the end of a word.
• Common three-letter words: THE, AND, FOR, ARE, BUT, NOT, YOU, ALL, WAS, ONE, HIS.
• Repeated word shapes. If two ciphertext words share a pattern — say ABBA or ABCABC — they represent plaintext words with that same pattern.
• Word endings. A three-letter ciphertext word repeated often is likely THE. Words ending in the same two cipher letters may share a suffix such as -ING, -ED or -TION.

A Worked Example

Here is a short ciphertext to try. Spaces have been left in place, which is generous of whoever encrypted it.

DTTZ DT QZ ZIT GSR DOSS WN ZIT KOCKT ZGFOUIZ QFR WKOFU ZIT DQH

Count the letters. In this 49-letter sample, Z and T both appear seven times, followed by D, I and O on four each. Z is a joint favourite, so try Z as E. But look at the word in third position: ZIT, which appears three times. A three-letter word turning up that often is almost always THE, which would make Z = T, I = H and T = E. That fits neatly with the counts, because T is the second most common letter — exactly where E should be.

Now the first word, DTTZ, becomes _ E E T. MEET, FEET, BEET — MEET reads most naturally for the opening of a message, so D = M. The second word DT becomes ME. Further along, DOSS has the pattern M _ _ _ with a doubled third and fourth letter, and MILL fits, giving O = I and S = L. The word GSR then resolves to OLD, confirming S = L and giving you G = O and R = D for nothing.

Keep going word by word, testing only the letters your table has already nominated. Where two candidates both make sense, leave the word and come back to it once a neighbouring word has surrendered another letter.

When the Numbers Lie

Frequency analysis is a tool, not a guarantee. Short messages are the main problem: under about a hundred letters, random variation swamps the signal, and a letter that ought to be rare can top the chart by accident. Themed text causes trouble too. A paragraph about buzzing bees will be unusually rich in Zs, and a list of exotic fruit will skew your counts badly.

Watch out as well for ciphers that are not simple substitutions. If you have worked out a mapping that mostly makes sense but a handful of words stubbornly refuse to resolve, you may be dealing with a transposition cipher, where letters are shuffled rather than swapped. Letter counts in that case still match English almost exactly — which is itself the giveaway. If your frequency table looks suspiciously normal, stop hunting for substitutions and start looking at word lengths instead.