Computer Cipher Solving – Lesson 7: cross-reduction

Last September I wrote a series of blog posts on computer cipher solving. They continue to get quite a few views, so it seems to be a popular topic. Today I’m resuming the topic by discussing cross-reduction of word patterns. This method has been used for centuries with simple substitution ciphers and it is still useful today for several types of ACA ciphers. Clearly it can be done without computers, but it is simple to program and a great time-saver. Often it’s the only way to break some tough ciphers. Don’t confuse this usage with cross-reduction in mathematics, which is a method to reduce fractions. The idea is to use the patterns of one word to reduce the possible solutions of a second word and vice versa.

The basic concept is simple. Take two ciphertext words enciphered with the same key, preferably long words, and compare the patterns to produce combinations of possible words that fit both patterns. It’s easier to understand by looking at an example. Headlines puzzles contain simple substitution ciphers, but they are often very short and have uncommon words such as proper nouns so usual methods like hill-climbing may not work. Consider this ciphertext from a recent ACA Headlines puzzle: EA’I NG THJLUZI IGIAHN AWBA’I MBCG. My hillclimber had no luck with it, but cross-reduction solved it easily.

Begin by reading through a word list and identifying all words with same pattern as THJLUZI, which is to say, all 7-letter words with no repeated letters. That sounds like a large list, but you can reduce its size by considering that Headlines puzzles use K3 alphabets to encipher. No, that’s not Kindergarten – 3rd grade. That K3 as used here: http://www.cryptogram.org/resources/cipher-guidelines/. Click on Keywords to get an explanation. The significance is that if a letter stands for itself in a K3 substitution, then all the letters must stand for themselves. In effect, the entire plaintext would be showing. Since it isn’t, that means none of the letters of the words stands for itself. That means you can eliminate all words beginning with T, having H as the second letter, etc. Next do the same thing with the word IGIAHN. Here, the 1st and 3rd letters are the same, so your list will probably be somewhat shorter. These lists may be saved in arrays or in files. Finally, you compare the two lists word by word to produce pairs of words such that there are no conflicts. That means the 2nd letter of word 1 must be the same as the 5th letter of word 2, the last letter of word 1 must be the same as the 1st letter of word 2, and so on.

The easiest way to do this with programming is to write a pattern function, one that produces ABACDEF for IGIAHN, etc. When you test every word pair (word1 and word2) from your lists all you have to do is ask if the pattern(word1+word2) = pattern(THJLUZI +IGIAHN). If it does, then display it or save that pair as a possible solution. My program produced about 200 pairs that fit the pattern. That didn’t appear to help much at first, especially since several combos seemed like plausible phrases from a newspaper headline (e.g. AEROBIC CYCLES). But notice that two words in the ciphertext end with an apostrophe I. That strongly suggests that I stands for plaintext S. Looking through my lists, there was only one word there that looked possible for the 2nd word: SYSTEM. The combos included DEVIOUS SYSTEM and LEPROUS SYSTEM, but it wasn’t hard to spot NERVOUS SYSTEM as the most likely candidate, and that proved to be correct. The solution is “IT’S MY NERVOUS SYSTEM THAT’S LAZY”. This method works best when you can find two words that have many common letters; if they’re adjacent, too, that’s even better, but not necessary. Needless to say, it only works if both words are in your word lists. When I get no solutions this way, I sometimes drop the final letter of one of the words in case it’s an inflected form (e.g. plural or past tense) and try again hoping the base word will be in the list.

This method will work with Key Phrase ciphers, too, even though that doesn’t use simple substitution. You just have to write a different comparison routine. Of course it works with Xenocrypts as long as you have good word lists for the language.

 

Venomous by Christie Wilcox

Venomous: How Earth's Deadliest Creatures Mastered BiochemistryVenomous: How Earth’s Deadliest Creatures Mastered Biochemistry by Christie Wilcox
My rating: 5 of 5 stars

I don’t read a lot of non-fiction, but I do enjoy a good science book. Wilcox has managed to turn a serious subject into a fun read and still impart the science subject matter in considerable detail. I enjoy learning something new, and almost every page held a fascinating nugget. I had no idea platypi, worms, and shrews were venomous. I didn’t know the difference between venom and poison. I didn’t realize how many different types of chemical attack were contained in the various venoms. All this and much more awaits you in this well-written exploration of our deadliest animal neighbors’ biochemistry.

The book is full of anecdotes, many lived by the author herself, depicting the bad luck, bravery, or idiocy (depending on your point of view) of those who have been bitten or stung, voluntarily or otherwise, by some of these creatures. She also recounts the efforts by clinicians and researchers to find ways of protecting humans from the effects of venom, and, perhaps more importantly, how to use venom to treat various diseases that afflict us. She does this with a scientist’s attention to detail and source citations. She provides an enormous lexicon of scientific terms. As a writer of murder mysteries, this book has filled my head with ideas (ah yes, “The Adventure of the Speckled Band” – it’s been done).

View all my reviews

Hide and Seek by Gary Slinkard

This delightful paean to geocaching will warm the heart of any geocacher. Gary’s boyish ebullience and complete devotion to geocaching are evident throughout the book. There is probably not much new for the seasoned geocacher, but the book covers a wide variety of geocaching-related topics, such as various resources on the web and elsewhere, how to negotiate with a non-geocaching spouse, what hazards to watch for while geocaching, and so forth. I’ve been geocaching since 2002 and I’d never heard of WVTim and his gadget caches until I read this book, so I think you’ll find plenty of nuggets to hold your interest even if you’re an old-timer.

For newbies and those who have never geocached but are curious about it, this is an excellent place to learn about geocaching or to broaden your knowledge and gain more enjoyment from the sport. Geocaching is much more than “using billion dollar satellites to find Tupperware in the woods” as the familiar slam goes. This book will show you why.

The author is not a seasoned writer, as he admits early in the book, and it shows. If you’re the type who cannot abide a split infinitive you’ll have trouble with this book, but I can only say that I just wish the average geocache page was as well-written. There’s an extensive glossary of geocaching terms at the end. The book can thus be a good reference resource for geocachers. Such terms are usually explained in the text as well, but at times they may be used before the explanation occurs, so if you come across such terms just jump to the back. The definitions are generally full explanations. There are also dozens of web links in the end notes so you can explore particular topics more fully. I like the inclusion of many photographs in the book, as well.

By way of full disclosure, the author mentions my own Cliff Knowles Mysteries in this book, a fact I didn’t know when I bought the book.

The Innovators by Walter Isaacson

The Innovators: How a Group of  Hackers, Geniuses and Geeks Created the Digital RevolutionThe Innovators: How a Group of Hackers, Geniuses and Geeks Created the Digital Revolution by Walter Isaacson
My rating: 4-1/2 of 5 stars

Isaacson has accomplished an impressive feat in summarizing 70 years of progress in the field of computing and the Internet. The 500+ pages may not seem like a summary, but he rarely spends more time than necessary on a topic. His central thesis is that innovators are most successful when collaborating with others. The brilliant idea man needs the practical manager to convert that idea into a useful product. He seems to agree with those who say that for most of even all of the great 20th and 21st century digital innovations, there were no inventors in the conventional sense, only people who contributed to bringing about such a thing. Every great innovation was incremental in nature and even the increments were collaborative processes.

Whether you agree with this thesis, you can enjoy his clear and concise well-researched account of each of the steps that led to the digital age we have. I was astounded at how many separate “inventions” have occurred in this time span, nearly all of which I have lived through. A look at the chapter titles will make clear what an incredible journey it has been: The computer, Programming, the Transistor, Microchips, Video games, the Internet, the Personal Computer, software, Online, The Web. Within each of these broad chapters there are a dozen or more key individuals who helped to bring about the concept and the reality listed in the chapter heading. The only major innovation he has omitted in my opinion is the cell phone/smart phone. I think this book must have gone to press just before the popularity of smart phones exploded.

I had expected this to be rather dry and boring, but it turned out to be anything but. This may perhaps be in part due to the fact I grew up in Silicon Valley. I was working in an IBM warehouse in one of my summer jobs in high school. I learned to operate mainframe computers for a work-study job in college. My first job out of college was operating such machines for a semiconductor company in Silicon Valley (then still known as the Valley of Heart’s Delight). In the 80s I bought an Apple ][e and taught myself Applesoft BASIC. When I switched to a PC I learned DOS and QBASIC, then Pascal and Delphi. I was on the high-tech squad in the FBI office in San Jose dealing with dozens of the high-tech firms mentioned in the book. I was offered – and took – the position of west coast security manager for AOL when that company bought Netscape. I was recruited away from there during the Dot Com Boom by another Internet company then laid off during the Bust. I’ve seen it all unfold before my eyes, but until now I never really had an understanding of who was causing this avalanche of change or why. This book has brought it all into focus and into perspective. If nothing else, it is a heart-warming nostalgia trip.

I listened to this book on CDs. The reader is excellent. I never got tired of his voice nor did he have the slick or smarmy sound of Scott Brick or other well-known readers.

View all my reviews

Our Ignorant Newsies – Len Ramirez edition

Last night Len Ramirez of KPIX news reported on a neighborhood that became completely surrounded by water from the recent flooding, in essence temporarily becoming an island. He described it as “landlocked.” Sorry, Len, that’s the exact opposite of what you meant. Landlocked means completely surrounded by land with no access to navigable water, like, say, Nevada or Mongolia. What you meant was enisled.

Before the Fall by Noah Hawley

Before the FallBefore the Fall by Noah Hawley
My rating: 4½ of 5 stars

A private jet goes down in the water somewhere between Martha’s Vineyard and Teterboro Airport. Scott, a painter and recovering alcoholic, is a last-minute addition to the passenger list, invited by Maggie, the wife of the wealthy news network owner whose company leased the jet. In addition to that executive’s family, other souls on board include their two small children, a money manager under investigation for laundering money for North Korea and others, a former Israeli war hero turned security expert, a pilot, co-pilot, and a beautiful flight attendant. Scott and the 4-year-old son survive in the water. Scott, who was also a standout swimmer in high school, must try to save himself and the boy despite a dislocated shoulder. He begins to swim, towing the boy.

The beginning of this book is riveting. Scott survives, that much we can predict from the opening, but who else does? Why did the plane go down? Weather was fine other than a low fog. The pilots were experienced and the plane had a good safety and maintenance record. We meet Gus, an investigator from the NTSB, and O’Brien, a jerkwad FBI agent. More bodies are found. Relatives come out of the woodwork. The post-truth news hounds descend.

This thriller borders on being great. The middle begins to drag a bit as the life stories of all the characters are brought out in a touch too much detail. The characters begin to take on cliched depictions. The NTSB guy is sympathetic and professional. The FBI guy is arrogant and suspicious. Cunningham, the cable news anchor, a sort of mix of Larry King, Rush Limbaugh, and Bill O’Reilly, is a lie-mongering viper. As we get closer to learning what actually happened, the pace quickens and suspense builds again. The ending does not come as a surpise since there are plenty of clues leading to it along the way, but it satisfies the hungry reader. Although the book is not perfect, it’s a solid four and half stars. I recommend it for those who enjoy thrillers.

View all my reviews

A Will to Die Cliff Knowles Mystery #7

The seventh book in the Cliff Knowles Mysteries series is now for sale on Amazon as both Kindle and paperback. You can also order the paperback at my Cliff Knowles Mysteries site.

San Diego is renowned for its sunny weather and beautiful beaches, but it has its dark side. It is the location of one of the largest mass suicides in American history. Thirty-eight members of the Heaven’s Gate cult left their bodily “vehicles” hoping for their souls to be taken aboard an alien space ship traveling behind the Hale-Bopp comet. Now, decades later, it looks like an offshoot cult is following the same pattern. One of the victims is the sister of private eye Cliff Knowles, a sister he hasn’t seen in years. He has trouble accepting the conclusion of the police that it’s a suicide. His investigation turns into a bittersweet family reunion when he comes to know his disabled niece whom he last saw when she was an infant. He’s determined to see justice done for her sake.

I Let You Go by Clare Mackintosh

I Let You GoI Let You Go by Clare Mackintosh
My rating: 4 of 5 stars

This book is told mostly in the first person, but not always from the same point of view. The scene switches from Jenna’s world to Ray’s (the Detective Inspector in charge of the hit and run case that resulted in the death of a boy). These two primary characters have their relationship problems (in their own worlds, not with each other). The setting switches from Bristol to a small Welsh village. There are a couple of big surprises in store.

What I liked the most about this story was the authenticity with which the writer treated the police work. Her twelve years working in the British justice system shows – and all for the best in my opinion. Although I know the U.S. federal criminal justice system thoroughly, the British system is quite different, a difference I found fascinating at times.

The ending is a bit too predictable and the characters just a tad too cliched for my taste, but all in all it was an engaging story that kept me wanting more. I listened to the CD audio version and enjoyed the accents. The actors, both male and female, are very good.

View all my reviews

A Man Called Ove by Fredrik Backman

A Man Called OveA Man Called Ove by Fredrik Backman
My rating: 3 of 5 stars

Few books can be summarized in two words, but this one is an exception: Lovable curmudgeon. That’s Ove and that’s the whole book. It’s well-written, blending humor and heartbreak. I enjoyed it despite its patent manipulative writing. The characters were all caricatures: Ove who could fix everything and physically overpower or intimidate anyone, his wife who could turn the most incorrigible juvenile delinquent into a Shakespeare scholar, the bureaucrat who was completely insufferable in every way.

I was irritated at times at this overwriting, and that probably dropped it a star in my rating, but I also have a more serious objection. Suicide, or attempted suicide, is not an appropriate subject for a whimsical or humorous tale, in my opinion. My family has been touched tragically in this way on more than one occasion and I can tell you it is the exact opposite of funny. This subject was introduced early in the book and I probably would have stopped reading at that point had the book not been the subject of my next book club meeting. Still, I’m glad I did keep on, because the plot moved on, as I knew it must, with most of the book remaining, and it really did bring some smiles and tugs at the heartstrings toward the end.

View all my reviews

So …

A new verbal tic has appeared — “So …” I’m not sure how or exactly when it came to be, but I’ve recently noticed that people  now begin the answer to almost every question with “So….” I hear this when people are interviewed on the local news. I hear this from panelists on radio shows or podcasts like “Tell Me Something I Don’t Know” or “Wait Wait Don’t Tell Me.” (That’s an interesting juxtaposition of shows, by the way.)

There’s something different about this one. Most of the verbal tics that have appeared in recent years have originated, or at least spread widely, by teenagers. I’m referring to “like,” “I mean,” “all” and similar annoying monstrosities as in “He’s like so stupid. I mean he’s all like ‘Why’d you do that?’ and I’m like ‘Whadda ya mean?'” Another one that bugs the hell out of me is the use of an objective case pronoun (or two) for the subject of a sentence as in “Me and him couldn’t agree.” Generally adults don’t fall prey to these.

This new one, however, seems to have taken hold in educated adults. At least it’s only at the beginning of each sentence and not scattered throughout, which makes it slightly less annoying. It still makes the speaker sound like an airhead, though. As a writer, I find it useful to identify these verbal tics so that I can immediately convey the idea that a character is stupid, lazy, or both by having them talk this way.

The word “so” has an actual meaning, several, in fact. As a conjunction it usually implies causation. “It was cold outside so I put on my coat.” As an adverb it suggests the extreme, similar to the word very, as “It’s so hot today,” but usually it implies causation in that use too, since one expects that phrase to be followed by a “that” phrase. “It was so hot that I had to remove my coat.” It can correctly be used as the answer to a question where that causation is implied. “Why did you put on your coat? So that I wouldn’t be cold.” But it’s not a general space filler. For example, “Q: Tell us a little about yourself. A: So, I grew up in New York …” I hear this constantly now and if you do it, you sound stupid, at least to word nazis like me. I’ve discovered I’m not the only one who has noticed this and deplores it, so, no, it’s not just me. It’s #SoAnnoying to a lot of people.

Google Ngram Viewer Christmas wishes

In my cryptography hobby I often use Google Ngram Viewer to try to guess what words might fit before, after, or even in the middle of a phrase. A few years ago I wrote a story using its predictive ability. I decided to try it again with a Christmas theme and the results are set forth below. Here’s how it works: I fed it the first four words of each sentence followed by an asterisk (e.g. Santa Claus is coming *). The Ngram Viewer responds with the most frequent words to follow that phrase in the millions of books it has scanned. I add the most frequent one to the end and remove the one at the beginning and repeat. So each word is produced by the previous four words. I rather arbitrarily chose where to stop and place a period so as to make a full sentence. So here’s the result.

Have a merry Christmas and a happy new year and many of them are not even aware of the existence of the world. Santa Claus is coming to town to see the doctor and the patient are in the same position as the first of the two. Joy to the world the first example of a new type of society that is not only a matter of time.

Merry Christmas and Happy New Year to everyone.

The Couple Next Door by Shari Lapena

The Couple Next DoorThe Couple Next Door by Shari Lapena
My rating: 5 of 5 stars

“Unrelenting” according to the blurb from Sue Grafton on the cover; I agree 100% I absolutely devoured this non-stop suspense thriller. Every page brought a new twist. Detective Rasbach is on the right track, but the track keeps changing. You’ll have to hold your head with both hands to keep it from spinning. There’s a confession. There’s a murder. Neither of those bring a resolution to the mystery.

Baby Cora is kidnapped while her parents are next door. All the character have flaws, we find out, but does that make them kidnappers? Both parents left Cora alone. Anne had too much to drink. Marco flirted with the sexy neighbor outside on the porch. The babysitter canceled at the last minute, but she used to fall asleep with her earbuds in. Would she have even been able to stop the kidnapper? Then we find that Marco and Anne both have secrets they’ve been keeping from each other.

This book has all the best qualities of Gone Girl and The Girl on the Train. Will Cora be found alive or dead? Will she be found at all? You won’t find out until the very end.

View all my reviews

Text-to-Speech – a new way to proofread

Here’s a tip for you writers out there. I’m trying a new technique for proofreading my latest Cliff Knowles Mystery: text-to-speech software. I opted for the free Panopreter Basic reader. Of course it produces a clunky computer voice, not a human-sounding one, but I expected that. If you use one of the paid versions, you have choices of voices and other features. This one delivers ads in the interface, but I ignore them. I don’t even know what’s being advertised since I never look at that window.

Since I write in Microsoft Word, my file has a docx extension. The reader requires another format, but it accepts several, including doc and txt. I use txt. Simply open up the application, click Input and Speak, and select your file from the menu. Your file’s text will appear in the main window.  Click the Speak button in the lower left and the voice (Microsoft Anna) begins to read. There are sliders to adjust volume and speed. Don’t set it too fast. It’s tempting to try to get through the proofreading chore as fast as possible, but you will miss things if you go faster than a leisurely pace. You are doing this to catch mistakes. I am amazed at how much easier it is to notice a mistake read aloud to me than it is to see it when reading. I’ve been catching a lot of errors this way.

The software is far from perfect. Getting used to the odd cadence of the voice is the hardest part. It will take some patience to use it. For example, if the word no appears at the end of a sentence, with a period after it, (like “She shook her head no.”) the software reads it as “number.” This makes for some funny dialogue. If a series of digits is four digits without a comma, like 2016, it reads it correctly as a date, and if it is preceded by a $ then it correctly recites the word “dollars” after the amount, but it pronounces Roman numerals weirdly and the word “is” always comes out as “eyes.” Since I write murder mysteries, the word homicide appears frequently. The software sometimes pronounces it right, but most of the time it comes out as “domicide” for some reason. There are many other anomalies. You can pause the reading to make a correction in your original file, but so far as I can tell, you can’t back up or fast forward to another location, at least not in the free version. If you shut it down, it will start at the beginning again. You can, however, save it as a .wav file and then you can navigate through that with any media player. I don’t bother with that. I just save my original in text files of ten to fifteen pages at a time and play them all the way through.

Still, you can’t beat the price, and it’s a lot faster than a paid proofreader, too. So expect my seventh Cliff Knowles Mystery, A Will to Die, soon. Unfortunately, it won’t be out by Christmas, but if you don’t get one of the items on your wish list for Christmas, you can treat yourself to this one in the new year.

Present Darkness by Malla Nunn

Present Darkness (Detective Emmanuel Cooper, #4)Present Darkness by Malla Nunn
My rating: 2 of 5 stars

Detective Sergeant Emmanuel Cooper is from the wrong side of the tracks – the South African tracks, that is. He is a white man living with a colored woman in the apartheid era. A white married couple is murdered and their daughter identifies two black students as the murderers. Cooper realizes quickly that the girl is lying but doesn’t know why. The white police lieutenant seizes on the identification as a quick solve of a heinous crime. It is up to Cooper to ferret out the truth.

The setting, both geographic and cultural, is esoteric and interesting to western readers; at least it was to me. However, that’s about all I can say this book has going for it. The plot was predictable and the characters hackneyed. Zulu detective Shabalala is ripped from the pages of the comic books – magically able to track anyone or anything, move absolutely silently despite near superhuman size and strength, and having that native intuition that can tell when anyone is lying. The doctor with them can cure anyone of anything with whatever is at hand – a shoestring, a jar of honey. Cooper’s inherent goodness can convince anyone to trust him 100% and cooperate with the police regardless of personal risk and can take any amount of beating or torture without revealing critical information. The bad guys are so obvious that they might as well have appeared with a neon sign on their heads from the first page where they were introduced and of course have not a single redeeming characteristic. I expected a “bwa-ha-ha” at any moment. There is no mystery here. You know how it’s going to play out from around page 10; at least I did.

View all my reviews

Mystereity Reviews Behead Me

I just got this really nice review of Behead Me by Tammy of Mystereity Reviews and had to share it. It’s nice to know that it appeals to non-geocachers as much as it does to geocachers.

29284829Behead Me by Russell Atkinson

Retired FBI agent Cliff Knowles is hired by a company to investigate possible theft of proprietary industrial plans, an investigation that leads him to shady business dealings, danger, and even murder.

Before you ask, Behead Me refers to a word puzzle. No blood and guts here, Behead Me is an engaging and absorbing mystery that will quickly draw you in and keep you turning pages. Cliff is an unflappable investigator, and I enjoyed how his methodology was neatly laid out. This wasn’t one of those super-cop-hunches kind of solves, it was a realistic procedural, and I really enjoyed how Cliff ferreted out the smallest details and connected them to another case, and the presidential pardon sub-plot was heartwarming and added some human dimension to the story. Cliff’s geo-caching adventure in the desert and the high speed chase at the end added a lot of action kept me on the edge of my seat.

I haven’t read the other books in the series, so I can safely say that this book can be read as a standalone, or as an entry into the series. The few references to prior books were minor and didn’t cause any confusion.

Overall, Behead Me is an intriguing and fascinating read and I definitely recommend it for fans of mysteries and procedurals.

I received a free copy of this book in exchange for my honest review

Behead Me is available at book retailers or online at Amazon  | Barnes & Noble

Series: Cliff Knowles #6
Rating:★★★★★ 5 stars
Publication Date: February 23,2016
Language: English
Formats: Kindle/Paperback
ISBN: 9781530039401
ASIN: B01C3QLX72
Genre: Mystery/Suspense
Reviewer: Tam (Mystereity Reviews

The Author’s Life

Adam@Home courtesy GoComics.com

Adam@Home courtesy GoComics.com

Sigh. How I sympathize with Adam. So far I’ve only had a typo on the cover once. Fortunately, it was on the back cover in the “About the Author” section. A sentence was repeated. If you have one of those copies, someday when I’m posthumously famous, it may be as valuable as those stamps with the airplane printed upside down. As for typos in the interior … I think they reproduce forever.