In 1840, Edgar Allan Poe wrote an article in the Alexander's Weekly Messenger, a Philadelphia newspaper where he challenged the readers to submit their own substitution ciphers which he would decrypt. Initially, he received cryptograms from around Philadelphia, but soon after, they came in from all over the United States. He published many of the cryptograms and their solutions in fifteen numbers of the Alexander's Weekly Messenger.

The next year, Poe published his essay called "A Few Words on Secret Writing" in Graham's Magazine, in which he commented on the response to his cipher challenge (see download below). The essay also gave birth to the famous quote that "human ingenuity cannot concoct a cipher which human ingenuity cannot resolve".

In the 19th century, most people considered secret writing and cryptography as a mysterious esoteric art, and Poe had sparked a great interest in cryptography among the general public. Thanks to Poe's publications, cryptogram puzzles became popular in newspapers and magazines. Inspired by the success of the cryptograms and the interest in his essay, he decided to write a short story that involved cryptography.

After writing The Gold-Bug, Poe submitted the story to a writing contest, winning the grand prize and $100. The story was published on June 21, 1843, in Philadelphia's Dollar Newspaper. It is regarded as the first important publication in popular non-technical literature that incorporated cryptography in its story line. The Gold-Bug contains a detailed description of how to solve a cryptogram using letter frequency analysis. The story was an instant success and helped popularize cryptography in the 19th century.

Edgar Allan Poe's The Gold-Bug is as iconic to cryptography in literature as David Kahn’s book The Codebreakers is to historical publications on the subject. The Gold-Bug became one of his most read and best-known stories. Many readers have set their first steps in cryptology after reading Poe’s story and some even became important codebreakers, such as the renowned cryptologist William Friedman. Many writers have been inspired by Poe to include ciphers and coded messages in their own books. Poe is also considered one of the pioneers of detective and mystery stories.

The Story

The main character is William Legrand. To escape some misfortunes, he lives at Sullivan's Island, near Charleston, South Carolina. Legrand discovers a brilliant gold-colored bug but lends it to someone else. When his friend, the narrator of the story, visits him, Legrand describes the rare bug with a death's-head on its back, and draws a picture of the bug on a piece of paper. A short while later, Legrand urges his friend to come back as soon as possible.

When his friend arrives, Legrand explains that he discovered a secret message in invisible ink after accidentally heating the paper with the drawing of the gold-bug. Having managed to decrypt the message, he invites his friend to join him on an expedition into the forest to search for a treasure near some rocks. Fearing that Legrand has lost his sanity, the worried friend agrees to accompany him.

The Gold-Bug is not only an exciting story about the discovery of an old treasure, but also a great introduction to cryptography and codebreaking. It tickles the reader's curiosity and Poe gave a detailed description of how to decipher the cryptogram and also provided the solution. However, deciphering the message yourself is even more exciting than reading how Legrand did in the story. Can I challenge you to decrypt the message, composed more than 160 years ago?

Decrypting the Message

Rather than just reading Poe’s story, we will show the technique and give you the chance to do it all yourself. It might be useful to read The Gold-Bug first, as the story might provide information that will help to solve the cryptogram, but only read to where the cryptogram appears! Don't cheat by reading any further for the solution. Searching the Internet for Poe or The Gold-Bug will also spoil the fun.

The message is encrypted by mono-alphabetic substitution, a cipher where each letter of the alphabet is replaced by another letter or symbol. We can calculate all possible he combinations for the 26 letters of the alphabet. The first letter is substituted by one of 26 symbols or letters, including itself. The second by one of the 25 remaining symbols or letters, and so on. The calculation 26 x 25 x 24…x 3 x 2 x 1 or 26! gives us a total of 403,291,461,126,605,635,584,000,000 different ways to replace 26 letters by another symbol or letter. How could we possibly decipher such a cryptogram? For centuries, substitution ciphers were regarded as unbreakable, but it is easier than it looks.

Although there are septillions of ways to allocate a set of symbols to letters, there are only a few ways to combine vowels and consonants in a natural language. Strict linguistic rules determine which letter combinations are possible and which are not. The syntax prescribes how words, and their order, are combined into a sentence, and conjugation rules determine how verbs are written. When we substitute letters with symbols, those symbols still follow all these rules and therefore create patterns that we can detect. Just as certain letter combinations are impossible (ZLG, XOJ, KFN,...), so will certain symbols avoid one another. Just as one vowel can only fit in a given set of consonants (THR?ST, L?GHT, ?NSW?R,...) so will certain symbols attract each other. But where to start?

The most important weapon to solve our message is letter frequency analysis, the basis of all codebreaking. Each language has its own typical distribution of letters in a text. In English, the letter E is by far the dominant letter, with an average of 12.7 percent. If we locate some of the most frequent vowels or consonants in the ciphertext or find the most frequently recurring symbols or combinations of symbols, then the rules of the language will give us strong leads to the words they are used in, or the letter they represent.

In the English frequency table below, the letters are sorted from most frequent left to least frequent right. Poe used an older and slightly different frequency table. However, frequency tables in any given language can differ, depending on subject, style and size of the text. Also, Poe did not use spaces, or a symbol for space, in his text. In plain text, the space occurs much more often than the letter E and would stand out clearly. To illustrate how distinctive letter frequency is, let’s count how often certain letters occur on this webpage: E=1163 T=887 A=630 D=320 W=158 V=70 Z=10.

To solve Legrand’s message, use a graph sheet to write down the secret text with symbols. Leave a blank row below each row of symbols, to write your solutions underneath the symbols in pencil (easily corrected with eraser). Next, count how many times each of the symbols appears in the cryptogram, and write down the symbols, ordered from most frequent to least frequent, underneath the letters of the English letter frequency distribution, shown in the table on the right. This gives a first rough indication of the frequency distribution of the symbols and the letters they might represent.

You will notice that one symbol clearly stands out. This is the first major clue, because the most frequent symbol represents without doubt the letter E, which is the most frequent letter in English. Write the letter E in pencil everywhere in the encrypted text, under the corresponding symbol, and also write that symbol underneath the letter E in the English letter frequency list. Make yourself also a second table with all symbols and corresponding letters you already found.

Next, try to spot recurring combinations of symbols. Some of the most used words in English, in order of frequency, are THE, OF, TO, AND, IN. Search for recurring combinations of three or two symbols that might represent those words. You should spot each THE quite easily. If so, you have discovered the solution for two more letters that are used frequently. However, not every combination of three symbols will represent THE or AND.

When completing words with an obvious letter, you will also be able to find or complete more words. If, for instance, you already identified the letters T and E, and you encounter fragment T?EE, it is not hard to imagine the missing letter R, especially when you already know the context. Vowel twins (AA, EE, OO…) are common but not that many different words contain such pairs. Try to find those words. Initially, it will be difficult to divide the text into words, but the more letters you solve, the easier it gets. If you can’t see it immediately, try various unidentified letters of the alphabet until you get something readable. When you assign a newly discovered letter to all its corresponding symbols in the encrypted text, this also helps you to reconstruct other fragments or locate words.

Be patient. It could take some time before the words appear in front of you, and some letter combinations might turn out to be incorrect. However, the more letters you do find, the faster you identify new letters or words. Before you know, you’re in a straight line to the finish. Letter frequency analysis is a powerful tool in cryptology and letter frequency tables for many other languages are available on the Internet.

Good luck...and make sure you don't get bitten by the bug!

Note: in the original edition, one symbol "(" was not printed near the end of the message, right after ‡9;48; although Legrand describes just that missing symbol to assist in finding a word. Since the story refers to that symbol, it is unlikely that it was omitted on purpose and probably got lost during the publishing.

Downloads

• Edgar Allan Poe's The Gold-Bug The original story (contains solution spoiler).
• A few Words on Secret Writing Poe's essay on cryptography with his comments on the 1840 cipher challenge.
• NSA - Signal Corps Bulletin No 97, 1939 Extract pdf p43-55, renowned cryptologist William Friedman on Edgar Allan Poe (full document).

More on Edgar Allan Poe and Letter Frequency (offsite, opens in new tab)

• Edgar Allan Poe's The Gold-Bug Digitized 1899 version Dana Estes & company, The Publick Domain Review (also at Internet Archive).
• Poe Museum Richmond Virginia Poe's biography, the complete works, educational resources and excellent blog.
• The Edgar Allan Poe Society of Baltimore The collected works of Poe.
• EdgarAllanPoe.nl Jean-Paul Colin's Dutch site with Poe's biografie, stories, poems and lots more. See also his @EdgarAllanPoenl YouTube.
• Edgar Allan Poe - Buried Alive PBS documentary on Poe's remarkable life and literary works (subtitles available in any language).
• The Gold-Bug Audio Book on YouTube Read by Vincent Price.

• Letter Frequency Statistics 14 different languages, sorted by letter and percentage, Wikipedia.
• Word Frequency Statistics The most common short words in the English language, Wikipedia.

More on Manual Ciphers

• Hand Ciphers Various techniques to encrypt messages with pen and paper.