The Spy and the Traitor: The Greatest Espionage Story of the Cold War by Ben Macintyre
My rating: 5 of 5 stars

This is by far the best book I’ve read in many months, perhaps years. It is outlandish and unbelievable, yet it has the added distinction of being absolutely true. It is the story of a KGB agent who spied for England. When I joined the FBI this is the kind of case I hoped one day to be involved in. It never came to pass, because this is the sort of case that happens once in a lifetime to an intelligence agency and only a tiny handful of people ever know about it at the time due to its absolute secrecy.

The beginning is well-written, but unexciting, setting the stage by introducing the characters and refreshing for us the realpolitik history of the Reagan-Thatcher-Andropov-Gorbachev era. I remember it well, but younger readers will find it eye-opening. By the second half of the book, the tension ramps up. PIMLICO, the mole, rises in the KGB and becomes more valuable, while at the same time the intelligence he provides puts him at greater risk as it is used. He is eventually put in peril. As it happens, western intelligence has its own mole betraying its secrets to the KGB. I’ll leave it there to avoid spoilers, but rest assured the last third of the book is a page turner. Do yourself a favor and don’t read through the photographs until the very end, as they contain some spoilers for what comes later.

View all my reviews

One of my favorite television shows was the original Law & Order starring Jerry Orbach as detective Lennie Briscoe. The writing involved complex legal issues in addition to the clever detective work. The acting and writing were excellent. It ran for 20 years, from 1990 to 2010. So I was excited to see it coming back starting tonight in Season 21. The various spinoff series were too gory or sensationalized with an unappealing set of actors. I never watched any of them for more than one episode.

Unfortunately, the reboot is poorly done. I’ve already taken it off my watch list. The writing is bad, the casting is bad, the acting is bad, the main characters are all unlikable, and it’s succumbed to the same problems as the spinoffs – more gore, more swearing, more overall nastiness. My wife called it ham-handed. I can’t recommend it.

In my last post, I mentioned key pruning. What I mean by that is halting the recursion (and allowing it to go back a level and continue with another word) based the nature of the keyblock. In particular, the block always has three rows. This means any given digit may have at most three letter equivalents. Thus I put a testing routine in the recursion routine to look for conflicts of this sort. If the word being tested passes the first conflict test, e.g. THEIRSOME in the last post, it then goes through a routine to count how many letters are represented by each digit. THEIRSOME is represented by 107735607. The 7 enciphers E and I while the 0 enciphers H and M. Since there are no cases where a digit represents four or more letters, this is a possible solution so far. This is not surprising since we’re only on level 2. If one of them represented four different letters, the recursion ends at that level, the test word SOME would be rejected, and the next word in the four-letter word list would be tried.

This example is so short that there are thousands of valid solutions (which is why Tridigital is a hobby cipher, not a real-life one). One my program found was “had some by their,” which even makes sense as a phrase. But with more words, especially longer ones, it will soon be the case for incorrect combinations that one or more digits represent more than three letters. This typically happens on the level three or four recursion.

The other technique I use is scoring. Since all solutions that pass through the conflict testing consist of valid words, the usual scoring techniques such as tetragram frequency or word list scoring don’t work. So I reorder each potential solution array back to its original order and test pairs of words for frequency.  I use Google N-gram data to determine this. The better the score, the higher in the display the solution is placed. Although this doesn’t speed up processing per se, it makes it much easier for me to spot a correct solution, or at least a likely correct segment in the solution early on. I don’t have to continuously scroll through dozens or even thousands of possible solutions as the program runs. The best ones are right up at the top.

As an example, the demo problem produces these possible solutions immediately:
had some by their
man some by their
arm some by their
had some up their
may some up their
man some up their

The most natural-sounding one is on top because the three two-word combinations (had some, some by, by their) score the highest in frequency. One of the other solutions may actually have been the first one found, but the display is in order of frequency score. Of course, we know none of these are the correct solution, but they are all valid.

This in turn allows another shortcut. If I spot a solution, or even a partial solution, I can abandon the program and use the digit-letter equivalencies revealed to run through my word lists in a separate program to identify any key words that can produce a keyblock with the same letters in the same columns, such as E and I in one column, H and M in another. Typically only about four or five words have to be solved before I have enough digits to find the original keyword and hat. Once I have those, I can solve any incorrectly solved or not yet solved parts by hand.

If you are not familiar with the Tridigital cipher, go to the American Cryptogram Association cipher types page and be sure you understand how it works. I’ll use the same keyblock on that page as an example.

N O V E L C R A F T
6 7 0 3 5 2 8 1 4 9
D R A G O N F L Y -
B C E H I J K M P -
Q S T U V W X Z - -

pt: t h e   i d e s   o f   m a r c h
CT: 0 3 0 9 5 6 0 7 9 5 8 9 1 0 7 7 3.

Note that every plaintext letter has only one digit that represents it, i.e. plaintext D is always enciphered with 6 in this case. The opposite is not true. The 6 can also represent B or Q. The spaces in the plaintext are always represented by the digit on the far right, 9 in this case. In the example, 0 is the highest digit and represents 10. The columns are numbered in alphabetical order according to the “hat” NOVELCRAFT. The ciphertext is a string of digits.

Step 1 is to find the spaces in the ciphertext, that is, determine which digit is the one on the far right. Since the plaintext will never have two spaces in a row, most digits can be eliminated by the fact they appear doubled in the ciphertext. For others, it is usually easy to judge by using the lengths of the words that would be produced by the possibilities. One might result in a 27-letter word, another in five different one-letter words. Those you can eliminate. This is just a preliminary step, not the heart of the algorithm.

Step 2. Once you have the ciphertext divided into word lengths with spaces inserted you need to prepare your word lists. Create arrays for each word length. To speed up solution I order the words by frequency, i.e. how common they are in English, but this is not necessary.

Step 3. Create an array of the ciphertext words, ordering them from longest to shortest, but keep a key so you can restore them to the order they appear in the original text. In the example, the elements would be 10773, 5607, 030, 58. In real problems, there are going to be some longer words, which is essential to an efficient algorithm.

Step 4. Begin testing words of the right length for the first word. Check for conflicts. Since WHICH is the most common 5-letter word, that would be the first one tested. We see the H’s would be represented by two different digits, 0 and 3, so it cannot be the first word. Go to the next word, THEIR in my list. There are no conflicts. The E and I may both be in the same column in the keyblock and thus both represented by 7. Save the word in a temporary array or string of possible solutions. Feed this word into recursive routine that is the key to the algorithm along with the level, i.e. 1 ion this case since this is the first word.

Step 5. Recursion. The recursive routine must first check to see if it has already completed all the levels. If it has, it has found a possible solution. That should be printed, displayed or saved in some fashion unless its score makes it untenable (I’ll discuss scoring later). If it’s not yet complete, start testing words from the next word list, the one with words the same length as the next level. In our example, that’s words of length 4 for word two, ciphertext 5607. In my list, THAT would be the first word tested and it would conflict with its own ciphertext (T would be represented by 5 and 7) as well as with the first word.  In my list, the first 16 words would conflict, landing on SOME as a possibility. The E is represented by 7 in both words, THEIRSOME. This combination is fed back into the recursion routine with the new level (2, here) and the process is repeated until a full solution is found or all the words on some level are tried without finding a possibility to send on to the next level. The trial words are then backed out of the temporary array or string and the next word at each level is tried until all the levels are tried back to level one where the process is repeated.

The experienced programmer, or cryptographer, will realize that the process as described above will eventually find the solution, but would take impractically long, especially with longer ciphertexts. The key to making this work is to find an efficient way to prune the words tried. There are two basic ways I’ve found, although there may be others. The two I use are key testing and scoring. I’ll save those for my next post.

If I Disappear by Eliza Jane Brazier
My rating: 4 of 5 stars

Sera is obsessed with a true crime podcast about missing women, young unattached women who disappear in Northern California. She’s the (now trite) unreliable narrator of the story. She’s a mess, after a divorce and having lost a child, the obsession is her life. Rachel, the podcaster, inexplicably stops podcasting and Sera is determined to find out why. She is convinced Rachel has been killed. She heads to Rachel’s remote ranch off the Klamath River where Rachel’s parents own a summer riding ranch for tourists. Everyone up there is weird and something clearly is not right, but she’s the outsider stirring up trouble among the hostile locals.

I was slow to get into this story. It is told in the first person, which is not unusual, but written as though Sera is talking to the missing Rachel, e.g., “I can’t believe you never told me. All this time you were trying to save others.” I’ll give it style points for originality, but I found it off-putting at first. The author has a knack for ratcheting up the suspense as the various characters shift in and out of the role of chief suspect. Is there even a crime? No one there has reported one. Her parents tell Sera she was killed by a gang, but everyone else says she just took off to get away from her crazy family. Sera’s parents hate and distrust the local townspeople and the feeling is mutual. None of the characters are likeable and there’s not a lot of meat to this story, but the answer as to what happened to Rachel (and possibly those missing victims in the podcast) is well hidden until the very end. I’d give it three and half stars if I could, but I’ll squeeze out four.

View all my reviews

In case you missed it, last Tuesday the far right wacko congresswoman from Georgia, Marjorie Taylor Greene, claimed that House Speaker Nancy Pelosi was sending gazpacho police to investigate the capitol riot. Apparently she doesn’t know the difference between cold soup and Nazis. This is surprising, since she is one of those. Apparently a lot of other people in this country also aren’t familiar with the meaning of gazpacho. Look at these two Google Trends charts. The top one is a comparison of how often and where the terms were searched on Google during the month of January. The searches were about the same frequency and randomly distributed. The bottom chart shows the searches this month. Nearly all of that blue searching has been in the last three days.

Greene is the same woman who claimed that Jewish space lasers were setting the forest fires in California last March.

It’s been a while since I last wrote about Varsity Blues, the college admissions scandal. Two defendants, both parents who bribed school officials to get their children admitted to colleges of choice, went to trial rather than plead guilty. Both were convicted last October. Yesterday the first of them, Gamal Abdelaziz, was sentenced to one year and one day in prison along with two years supervised release after prison, a $250,000 fine and 400 hours of community service. The other defendant, John Wilson, will be sentenced next week. He was convicted of tax charges in addition to the college conspiracy charges, so he may get an even longer sentence.

These two men are the first two to choose trial rather than taking a plea deal. It cost them at time of sentencing. The longest sentence given out prior to this was for nine months. That defendant, Douglas Hodge, paid more money and was more directly and actively involved in the bribes on behalf of four children than most other parents. There have been several other defendants convicted in recent months, too. To see the whole list, here’s the authoritative link: Varsity Blues.

The most outrageous case is the one defendant who was pardoned by Donald Trump right before he left office. Robert Zangrillo, a Miami developer, was let off scot-free. The White House claimed that various businessmen and even a USC trustee (a major Trump supporter), advocated for the pardon for Zangrillo, but those businessmen all denied having any input on the pardon or contact with Zangrillo or the White House. The White House statement on the pardon also claimed that Zangrillo’s daughter Amber was earning a 3.9 GPA at USC, but the school confirmed that she was not enrolled there. The lies from that man never end.

The Last Flight by Julie Clark
My rating: 4 of 5 stars

Claire Cook is married to the son of a powerful U.S. Senator from a power family (e.g. Kennedy, Bush). But her husband is controlling and abusive, so she decides to escape. She formulates a plan, but the plan is thwarted. Instead she ends up switching plane tickets with Eva, a young woman also in a desperate situation. The stories of the two women are told in interspersed chapters, Claire’s moving forward, Eva’s in flashbacks leading up to the day of the ticket switch.

I really like the premise. It’s a thriller, not a crime novel per se, although there are crimes involved. Tension was kept high. Will Claire’s husband find her? What happened to Eva? The writing was rather uninspired and won’t win any literary prizes, but it featured some locations I’m familiar with, which always adds to my interest. The ending wasn’t great, but it did explain one mystery and contained a surprise, so it wasn’t a disappointment.

View all my reviews

This will be my last post on the subject of Wordle for a while (I hope). I wrote a program that tries to guess the input Wordle word. I’m calling it Worbot. I wrote it so that its attempts are shown in the same format as the actual Wordle. The program doesn’t have access to the Internet, so it can only solve the word that is input. It does a pretty good job of it. The only word Worbot hasn’t solved in six tries (the maximum allowed by actual Wordle) is PROXY, which is also the only word I didn’t get in six tries. And I had to run the word PROXY three times before the program failed. It rarely fails to solve within four guesses. Here are some sample runs. Spoiler warning: one of the words shown is today’s actual Wordle word, but you probably won’t read this today.

My purpose in writing this is to see if I can beat my program over time, i.e., develop a better record. I don’t yet have the routine written to track that for the program. My word list is slightly larger than the one used by Wordle, so some of my words may be invalid, but it could also be that Wordle uses some words not in my list. It may be possible to download the Wordle list, but I’m not concerned enough to go to that trouble.