About Hand Ciphers Hand ciphers, also known as pencil-and-paper ciphers or field ciphers by the military, are encryption methods that are performed with nothing more than pencil and paper. Many of these classical ciphers were developed in two-thousand years of cryptography. With each new cipher that was developed, codebreakers found new attacks to break them, and codemakers improved their ciphers and devised new methods to elevate the security of their ciphers. Hand ciphers have always been a compromise between security and practical considerations. Very secure ciphers were often very elaborate, unpractical and prone to errors. This was a disadvantage in tactical circumstances. On the other hand, practical and less complex ciphers were often weak. Nonetheless, these insecure ciphers proved to be interesting in some tactical circumstances where time to break them would exceed the tactical life span of the message. On this page you will find some methods and techniques that are the result of many years of evolution in cryptography. I won't go through the simple types like Caesar Shift, ROT-13 or all kinds of Vigenère variations that are easy to break with nothing more than pencil and paper but will focus on the more complex ciphers that earned their stripes in the 20th century. Some of the explained techniques can be useful to develop your own hand cipher. One-time Pad The one-time pad encryption is a special kind of cipher that has its own page on this website. Please visit the one-time pad page for more details. Double Transposition The double transposition or double columnar transposition was probably the most secure and most popular field cipher during the Second World War. It was used by both Allied and Axis forces and a regular change of good keys ensured a very good resistance against cryptanalytic attacks. At the same time, it was rather easy to apply, although time consuming for long messages. One disadvantage is that letter frequency analysis will show a normal linguistic distribution, similar to plaintext, which points to transposition. When enough ciphertext available, encrypted with the same key, an attack by anagramming could be successful, although very difficult. A very effective way to beat the anagramming is to first fractionate the characters of the plaintext (see further). The message to encrypt:
You can separate words or sentences by an X if confusion could arise. The text is completed with X's until the number of letters in the message is a multiple of 5 (spaces not counted). The keywords: ALADIN and CONSPIRACY Now we can make our first transposition. Create a matrix and write the first keyword in that matrix. Beneath the keyword we write the sequence of how we will read off the letters. Assign 1 to the keyword letter that is first in the alphabet. The second letter 2 and so on. If two identical letters occur in the keyword the most left letter will get the lowest digit. The first transposition matrix:
Next, we create a second matrix with the second keyword. Again, we assign digits according to the order of the letters in the alphabet. We then read off the letters from the first transposition matrix column by column according to the key order sequence and put them row by row into the second matrix. The second transposition matrix:
The final ciphertext is again read off column by column according to the key order sequence of the second keyword. The final ciphertext in groups:
To decipher the double transposition, we work in exactly the opposite direction. First, we create a matrix with the second keyword and determine the long and short columns (free places at the end) according to the message length. The ciphertext is written into the matrix, column by column, according to the keyword sequence. Next, we create the matrix for the first keyword, also determine the long and short columns according to the message length and read off the matrix from the second keyword row by row and write it into the matrix of the first matrix, column by column, according to the keyword sequence of that matrix. Finally, we read off the plaintext row by row. Note that in reality, much larger keywords or key sentences, 20 letters or more, are used to enable the encryption of large pieces of plaintext. Disrupted Transpositions
The disrupted transposition is as a further complication to the normal transposition. Instead of filling the matrix row by row, the rows are all filled in a very irregular fashion, resulting in two separate areas. This results in a very complex transposition of the characters. First, we determine the exact number of rows and columns to fill (don't forget to complete with X's until we have a final group of five letters) Next, we fill a row until we reach the digit from the keyword sequence. If the first digit is at the 8th place, we will only fill that row up to that position. We continue the next row until the second position and so on. If we have reached the end position of the last line, we continue by filling the remaining empty places at each line. In our example the difference between the two areas is visible by the lower- and upper-case characters. The plaintext:
We use the keyword CONSPIRACY On the left, we see the matrix after filling the first area, and on the right, we see the same matrix filled completely:
Once the matrix is filled we read it off by the columns, according to the keyword sequence and then rearrange in groups.
To decipher a message, you first reconstruct the width and height of the matrix according to the received number of ciphertext letters, to determine the number of trailing empty cells. Next, mark the two separate text areas. Then fill the grid with the ciphertext, vertically, column by column, according to the keyword sequence. Finally, you read off the plaintext in the first area and then the second area. There are several types of disruption possible. Another possible method is that of large triangles. Each triangular area starts at the first keyword position and goes one step further to the right with each lower row. In the example we can observe two triangular areas. Again, we read off by the columns, according to the keyword sequence. As you can see the second method will result in a very different ciphertext.
Usually, the disruption method is used in one of the two transpositions of a double transposition, making it much stronger. Probably the most powerful German wartime hand ciphers was Rasterschlüssel 44, a grid-based cipher that disrupted the text rows by a grid, followed by columnar transposition. Fractionation Ciphers that are based purely on transposition reveal statistical information when subjected to letter frequency analysis. Breaking up or fractionating letters before moving them around improves the security of a cipher considerably. A simple matrix can break up letters into a horizontal and vertical coordinate. This Polybius square is the easiest way to apply fractionation on plaintext. A scrambled alphabet can be used to complicate the matrix. Of course, fractionation offers no cryptographic strength unless combined with other techniques.
The Polybius square on itself is a very weak mono-alphabetic substitution cipher. The Russian Nihilist cipher used a keyword scrambled Polybius square where the plaintext digits are added to a fixed number of digits, derived by a keyword from the square. It was basically a pimped Vigenère encryption and simple to break. Fractionation should only be used in combination with other techniques. Always fractionate the letters before transpositioning them, preferably an irregular fractionation followed by a double (disrupted) transposition. This combination is one of the strongest manual ciphers. The Straddling Checkerboard
An important Russian improvement to the fractionation of the Nihilist cipher is the straddling checkerboard. Letters in a normal matrix are broken up into ever equal parts. The checkerboard however produces a most irregular pattern. Some letters are represented by one digit, others by two digits. By assigning the most frequent letters to one-digit values the resulting ciphertext will be smaller than a normal Nihilist cipher. Also, the irregular pattern of digits makes it much more difficult to retrieve the matrix combinations. As a further complication one could fill the matrix with a scrambled alphabet instead of the default alphabetic sequence. The combination of fractionation and transposition proved to be very secure. A good example of such a combination is the VIC Cipher with double transposition and straddling checkerboard, used by Russian intelligence during the Cold War. The cipher resisted all cryptanalytic attacks for several years until the Soviet agent who used it defected to the United States. A straddling checkerboard matrix, with one-digit values optimized for the English language:
To convert figures, we use "FIG" before and after the digits and write out each digit three times to exclude errors. Let us convert some text with the checkerboard
In groups, completed with zero's:
It's easy to convert the digits back into characters and separate the one-digit values from the two-digit values. If a digit combination starts with a row number (2 or 6) it's a two-digit code and another digit follows. In all other cases it's a one-digit code. Again, this fractionation method should always be used in combination with other cipher techniques such as transpositions or a one-time pad. After converting the text into digits one can transposition them or add a one-time pad. If used with a transposition, we simply transpose digits instead of letter as we do with the double transposition cipher. Remember, a checkerboard on its own does not provide any encryption nor security. Always use it in combinations with some type of encryption. You can also construct a straddling checkerboard with scrambled alphabet that is still letter frequency optimized and based on an easily remembered keyword. Let's use the keyword SCIENTIFIC. Write out the keyword letters but omit the repeating letters. Add the remaining letters of the alphabet below the keyword, skipping the already used letters. Read off the scrambled alphabet column by column (there are many more methods to create scrambled alphabets).
Next, mark as many of the most frequent letters as you need to fill the top row of your checkerboard. They can be chosen from the list ETAOINSHRD, which is written out in English frequency order. Let's pick the easily remembered letters AT ONE SIR.
Let's also derive the order of the digits from the keyword. Assign the digits in alphabetic order with zero as last digit. For identical letters, the leftmost one is the first to receive a digit.
For our example, we fill these in a standard checkerboard. Write the marked letters in the top row as you find them in the scrambled alphabet and then fill in the rest. The series of digits are also placed in the checkerboard, leaving the two last cells blank. There you have it, a straddling checkerboard, based on the keyword SCIENTIFIC. Another small obstacle for the codebreaker.
Extending the Straddling
Checkerboard The number of one-digit values determines the possible number of rows and therefore the number of supported characters. The CT-46 table is an example of an extended checkerboard. It is optimized for use in different languages. Other variations with different character sets are possible. Another checkerboard variation is TAPIR, used by the former East-German intelligence, optimized for German language with special German characters and even syllables. The CT-46 table:
On the CT-46 table, all frequent letters in the top row have one digit, taken from the columns (A=1, E=2 etc). All other characters are a combination of a row digit and a column digit (B=70, C=71, W=86, 2=02 etc). More conversion tables are found on the checkerboard variations page. More Fractionation The Polybius and checkerboard matrices cut your characters in two pieces. There are however methods to divide them even more, which makes it harder to get them puzzled together by the attacker. The Trifid cipher works with 3 matrices with 9 characters each. Each character is converted into 3 digits: one for the matrix, one for the row and one for the column. Of course, we can fill the three matrices with a scrambled alphabet. The system is easy to expand. With three matrices of four by four we can already support 48 characters. The Trifid fractionation:
The plaintext converted:
As with all fractionating ciphers, this should always be used in combination with some sort of transposition. However, a cryptanalyst would always instantly see that the ciphertext contains only the digits 1, 2 and 3, and will understand that at least one stage of the encryption is a Trifid type cipher. Without transposition, he could cut the ciphertext in pieces of three digits and treat them as one character of a mono-alphabetic substitution, which is very easy to break. But there are ways to mislead the attacker. We can assign several letters to each of the three values and randomly pick out one. We need a table that says which letters stand for which value. We could fool the cryptanalyst even more. If we pick out frequently used letters such as ETAOIN more than other letters, we can create the illusion that it's not a fractionated cipher but a simple transposition, since the letter frequency analysis shows a normal distribution. The attacker doesn't know that each letter isn't a letter, but just one third of any other letter. We give the digits some letters. You could use a scrambled alphabet. If you consider extending to three matrices of four by four you must take in account that digit 4 is used slightly less than the other digits (the first digit of a character will never be 4 since you have only 3 matrices). In that case it's better to assign a few less letters to digit 4 to compensate the unequal distribution (less letters to a digit means that the individual letters are used more often than when you had more choice for that digit).
This is the new result, with a random letter for each of the 3 digits.
To decipher this, just see which value a letter is assigned to and you can recompose the digit combinations. Note that a character can be represented by many different combinations of three letters. In the example we see that the letter E is converted in both NAH and WYE. We can also observe that the ciphertext letter A is one third of the plaintext letter Y, but also of E and T. Even without combining it with transposition, it would be very hard to find out that Trifid is used, and even if we knew, letter frequency analysis is useless. In combination with transposition, we could use one keyword to scramble the matrices and the digit-letter combinations and one or even two keywords for a single or double transposition. A very hard nut to crack. Let us take this one final step further. Four matrices, each a mini straddling checkerboard! The irregular result is a code of two or three digits, just as the straddling checkerboard, but cut in two or three pieces instead of one or two. This means more fractionation. The first digit of a code is the matrix number. If the letter is in the top row, it will be a two-digit code. If the letter is in the bottom row, it will be a three-digit code. In the example I used a scrambled alphabet, easy to make with a keyword, as explained in the Keywords section. The top rows of the matrices contain the top 12 most frequent letters (in this case of English). These are E T A O I N S H R D L C. It's not hard to fill the matrices with a scrambled alphabet. You first mark the top 12 and fill them one by one in the top row. Next, you fill the rest of the letters into the bottom row. Again, the combination 444 is used to switch between letters and numbers.
Here's the scrambled alphabet I used for the matrices:
And here we see the Quadratic Checkerboard, or whatever you want to call it because I just invented that name, in action:
Of course, we have to convert the digits into letters, and this time we need a table with the letters divided over four digits instead of three. After a few transpositions, I wouldn't want to break this one! Digraph substitution Digraph substitution is another excellent way to prevent single-letter frequency analysis. Each two-letter plaintext combination is replaced by a digraph from a secret table. There are various ways to use digraphs with varying degrees of security. The advantage of digraphs is that each plain letter of a pair can be enciphered into many different letters, depending on other plain letter in that same pair. One of the first ever digraph ciphers is Playfair, which requires a secret scrambled alphabet. Let’s create one from the keyword FIRESTORM: Write out the keyword letters, omit the repeating letters. Next, you add the remaining letters of the alphabet below the keyword, skipping the already used letters, and omit the letter J (since our Playfair grid only has 25 cells, we can use the letter Y for both Y and J). Finally, read of the scrambled alphabet by the columns, from left to right.
Read off by the columns, left to right, we get:
We put this alphabet in a 5 x 5 square.
Three different situations can occur when encoding a digraph. Let’s give some examples.
Since the Playfair cipher cannot encrypt repeated letters (aa, kk,...) we always must insert an X between those letters. let’s encipher the message ”report troops by shortwave radio” with our scrambled alphabet:
Note that we inserted an X between the two O’s to avoid repeated letters in a single digraph. An incomplete last pair is completed with an X.
To decipher a Playfair message there are again three possible situations. If the digraph letters are in different rows and columns, then use exactly the same process as during enciphering. If the letters are in the same row, then reverse the process and replace each letter with the letter to its left instead of right. likewise, if the letters are in the same column, then replace the letter with the letter just above it instead of below. We can see that letter a can be enciphered into Z or N, letter s into E or G, and letter r into B or P. In fact, each plain letter could be enciphered into many different letters, depending on the other letter in that pair. This is an excellent feature. There are however some important flaws in how the Playfair cipher works and how it is applied. A disadvantage to Playfair is that the reverse of a plain letter pair enciphers in the revers of its digraph, as shown with rt enciphered into PH and tr into HP. Also, Playfair cannot encipher a letter pair containing two identical letters. These problems can be solved by the Two-square or the stronger Four-Square cipher, as shown below. Another issue is how the text is enciphered. if we take the digraphs in plaintext order (re, po, rt ...) then we merely replace the plain pairs by their digraph counterpart. As with a monoalphabetic substitution, simple frequency analysis of digraphs easily identifies the digraphs that replaced often used plain pair combinations in the text, like AN, EN, ON, TH, TO and so on (most common bigrams in English). This enables the reconstruction of plaintext and scrambled alphabet. This can be countered by splitting the plaintext in halves, write the second part underneath the first and read the plain pairs vertically. By doing so, you create distance between the two letters of the plain pairs an break up common plaintext pairs and their linguistic relation. Finally, although digraph systems prevent single-letter frequency analysis, they are actually more a code than a cipher, as they always replace the same plain pairs by the same digraph ciphertext equivalent. The use of a single, or preferably double transposition after a digraph substitution will break up any relation between the pairs. Let’s see how Four-square works. We need two normal and two scrambled alphabets. We use the keywords INTRUDER and COUNTERATTACK. We again omit repeated letters and read off column by column. This time, we simply use an alphabet without the letter Z, the least used letter in English. You could however select any other letter.
We now create the Four-square with the two normal alphabets without the letter Z, and two scrambled alphabets, also without the letter Z. You could use any another letter, but always omit the same letter for all alphabets and make sure all correspondents use the same alphabets with the same omitted letter.
Our plaintext message: report troops by shortwave radio. We take the plaintext, divide it in two halves and write the second half underneath the first. If required, add a letter X at the end. Read off and encipher the plain digraphs vertically, column by column. To encipher, thanks to the four squares, we can always apply the method of rectangles with the digraphs in opposite corners. The first plain letter is found in the top left alphabet and the second plain letter in the bottom right alphabet. Their digraph counterparts are found in the opposite corners of the rectangle that is created by the two plain letters, with the digraph letter in the same row as its plain counterpart. rs becomes PX, eh becomes KN and so on. Notice that plain eh is enciphered into KN, and its reverse he is into CP. Playfair would have simply reversed the enciphered digraph.
To decipher the message, the first digraph letter is found in the top right alphabet and the second digraph letter is found in the bottom left alphabet. Their plain counterparts are found in the opposite corners of the rectangle that is created by the two digraph letters, with the plain letter in the same row as its digraph counterpart. However, the obtained plain letter pairs are written out vertically, followed by the next letter pair, thus recreating the two half parts of the text underneath each other. Many improvements are possible. The two normal alphabets for a Four-square can also be replaced by scrambled alphabets. You could use a 6 x 6 grid (36 cells) allowing the use of the full alphabet plus all digits. One or two additional transpositions would greatly strengthen the cipher. Our final example to apply digraphs is a full 26 x 26 table with completely random digraphs. This is the most secure version. Only a part of the table is shown for this example. The advantages of such a table are the possibility to encipher repeated letter pairs and a security that is far superior to any small table that uses geometric methods to compose digraphs.
The downside of a 26 x 26 table is that you need to create two tables, one to encipher and one to decipher. You could avoid the use of two tables by making the table fully reciprocal. In that case, the cross section of AB would for instance contain FQ and the cross-section of FQ would contain AB. This solution will affect the strength of the system, but it will still offer far better security than Playfair and alike, and the reciprocal system is much more practical. Another disadvantage of a 26 x 26 table is that you cannot compose all the random digraphs with simple means like passwords and you cannot memorize such a large table. It is important to understand that all digraph systems become more and more vulnerable when the number of messages, enciphered with the same digraph table, increases. A few messages, enciphered with a random 26 x 26 table, could resist all cryptanalysis, while tens or hundreds messages, enciphered with the same table will easily succumb to the codebreaker. Regular change of digraphs is essential. But, as said before, digraphs, incorporated into other cipher systems, can increase a cipher’s security considerably because they prevent single-letter frequency analysis. Improving Keywords There are several methods to scramble an alphabet with keywords or to stretch them. The most common way to create an alphabet for use in a matrix is by mixing an alphabet in a matrix with one or two keywords. In our fist example we use the single keyword ENTERPRISE. First, set up a matrix as wide as the keyword, but use each letter only once. Now, there are two ways to fill the matrix completely. The first method is to fill the matrix with the remaining letters of the alphabet. The second method is to also fill in the remaining letters but leave a space for each letter that has already been used. The actual mixed alphabet is read off column after column, by the order of the top letters. The two ways to fill the matrix with a single keyword:
The key alphabet result of both methods, read off by the columns. Notice the different results:
Let us now create a double keyword alphabet with the words INDIGO and ENTERPRISE. First, set up a matrix as wide as the first keyword and use that keyword as column header. Next, we fill the matrix with the second keyword, using each letter only once. We can now complete the matrix with the remaining letters of the alphabet. Again, we can choose between filling continuously or with spaces. The actual mixed alphabet is read off column after column, by the order first keyword. In this example we see the same double keyword with the two ways to fill the matrix:
The key alphabet results of both methods read off by the columns:
There are many variations possible to fill the matrix. One could fill them in a spiral from the outside to the inside, clockwise or counterclockwise, from right to left and so on. Your imagination is the limit. These scrambled alphabets can be used to fill Polybius matrices or checkerboards, or to create complex transposition sequences. Generating random digits
An easy way to create random digits is the Lagged Fibonacci Generator. It is a method also called chaining. Note that this is by no means a cryptographically secure way of generating numbers, and the output randomness depends highly on the initial state and size of the generator. However, the chaining method can be useful as a means of super-encryption or to individualize an encryption scheme. To perform a chaining, we take add first and second digit together modulo 10 and append the result at the end of the array of digits. Next, we take the second and third and so on. The more distance between the initial and the generated digits the more complicated the relation between initial state and generated digits will be. Example of a 10 Lagged Fibonacci Generator (the first row is the initialization).
Of course, we can vary the size of the initial state, change the distance between the two digits that we want to add, or use another modulo value. This generator can be used to improve resistance to cryptanalysis of encryption types such as double transposition. We know that a high number of ciphertext messages that are encrypted with the same key will help cryptanalysts to break double transpositions by means of multiple anagramming. We can apply the usual keywords to create the transposition matrices but change the final transposition sequences for individual recipients by adding a secret series of digits. This could be the recipient's personal number, a birthday or easily memorized number. Although all users have the same keys, they all have another transposition result, and the statistics guys have less material with the same key. Even a scheme with a real individual message key for each message could be developed. As an example, we chain the personal
number 54826 until we have 10 random digits and add them
to a normal column header of a transposition matrix. The
result is a completely different transposition.
An example of a transposition that is scrambled by a key number chain (we don't use the initial state digits). The new transposition sequence is taken from the sum where the 1 is the first, 2 the next and 0 the last. When identical digits occur in the sum, the most left one is to be taken before the other.
Super-encryption Chaining a secret number can also be useful as super-encryption. Super-encryption is a second encryption on top of a normal encryption. This offers additional complexity to an existing encryption. One could use it to super-encrypt a one-time pad ciphertext. In case of a compromised key, it would not be possible for the attacker to simply use the one-time pad to decipher the message. The result would appear to be a random series of values and make them believe they have the wrong one-time pad. In such a case, the chained digits would act as an additional protection, next to the physical protection that is required with one-time pads. Let us take a one-time pad with simple conversion A=01 to Z=26 where encryption is performed by subtracting. It is super-encrypted with the personal number 54826. The personal number chained:
A one-time pad super-encrypted with the key number chain (subtraction is performed without carry):
An example to explain multiple subtraction modulo 10 without carry: 2 - 4 - 9 = 9 Because 2 - 4 = 12 - 4 = 8 and 8 - 9 = 18 - 9 = 9 To decipher this one, just recreate the chain with the personal number and add the ciphertext, one-time pad and chain together without carry. The resulting digits are simple converted into plaintext letters. More about one-time pad on this page. Knock-Knock An interesting variation on the Polybius square is what I call Knock-Knock, commonly known as Tap Code. It allows a most basic form of communication. Where other simple communication methods like Morse need at least a light beam or have to produce sound to transmit dots and lines, Knock-Knock requires only pulses that can be transmit in all kinds of forms, knocking being the simplest of all. Note that this method is only a substitution cipher and therefore very easy to break. It should be seen as a method of transmission, rather than encryption. A simple Tap Code matrix:
Each transmitted code is formed by the combination of rows and columns. According to the matrix, the letter M is converted into 32. To transmit, we give 3 knocks, a small pause and 2 knocks. Take a larger pause between the letters. In the example each X represents a knock with whatever you have, on any surface (don't use your head!). You could use it when your submarine is stuck on the bottom of the ocean, when you're trapped underneath a building after an earthquake, or to ask another inmate in your favorite prison to pass the metal saw after he's finished (a disclaimer could be useful here).
We can create a Letter frequency optimized Tap Code matrix. To reduce the number of 'knocks', required to send the characters, we can assign the smallest combinations to the most common letters. The six most frequently used letters present about 50 percent of text and assigning them short combination will reduce transmission time considerably.
Conclusion A secure cipher should have keywords that are easy to remember, is easy to apply without errors and offer a good security. It should use substitution, transposition as well as fractionation to resist cryptanalysis. The user must determine whether he focuses on security, applicability or speed. The encryption techniques as described above provide the latest and strongest encryption combinations before the era of digitalization in cryptography and can be a guideline to the development of your own encryption scheme. Many combinations, extensions and adaptations on the above explained techniques are possible. However, although attacking some of these ciphers requires extensive and complex cryptanalytic techniques, modern computational power is able to break them by brute force (with the exception of one-time pad). Nonetheless, these ciphers still require good knowledge of cryptography and/or software design to be broken. A ciphertext, produced by an unknown secret encryption scheme, could even resist all modern techniques when the attacker can't determine the nature of the algorithm. As always, cryptography is a risk-analysis, a balance between the efforts and costs your protection is worth to you, and the amount of effort, time and costs the potential attacker is willing to spend to break it. Finally, modern cryptography of course provides stronger and faster software algorithms, and hand ciphers are to be considered as replacement in case of unavailability of computers or other encryption devices. More on this Site
Off-site Links (offsite, open
in new tab)
|
