Monthly Archives: September 2018

What3Words on the News

Once again it’s time to see what we can learn from the What3Words site about today’s news. There we learn that Trumps.next.nominee is totally at sea as is the Senate.judiciary.committee. A strange.hearing.ensued in Quebec, followed by a party.line.vote near San Antonio, Texas. Now it turns out that Flake.wants.investigation by the FBI in South Australia before a full.senate.vote to occur in the desert of western China. I’ll never. understand.politics. Hmm, interesting location for that one … maybe it explains a lot.

Unhinged: An Insider’s Account of the Trump White House by Omarosa Manigault Newman

Unhinged: An Insider's Account of the Trump White HouseUnhinged: An Insider’s Account of the Trump White House by Omarosa Manigault Newman
My rating: 3 of 5 stars

As others have said, Omarosa is not an admirable character. She’s self-serving, narcissistic (like guess who), and to some at least a sell-out to her race. She also writes very poorly; more on that later. The book will be judged largely on the reader’s political bias, and there’s little to be said about that. But one thing I learned as an FBI agent is that just because a sleazeball is telling you something, that doesn’t mean it’s false. Some of my best, verifiable information came from scummy informants or from defendants who turned on their pals to avoid jail. The book spends way too much time discussing her childhood, her rise to riches through TV, and so forth, nearly all of it portraying her as some sort of poor girl made good through hard work (and winning beauty contests). When she gets to Trump, her accounts really don’t give much that’s new. She describes him pretty much the way he appears on TV – rambling, constantly contradicting himself, attacking others who have not been loyal to him at least in his view, lusting after women including his own daughter, spouting racist language (Mexicans are murderers and rapists, etc.) If she wanted to lie and dump on him, she could have come up with stuff beyond what he himself has done and said publicly. Her main criticism of him, if you want to call it that, is that he is in mental decline. It’s clear as she states at the end, that she still cares about him and considers him her mentor, the one who raised her to fame and riches, even though she recognizes his racism, not only against blacks but also against other minorities like Jews and Puerto Ricans. She is definitely vengeful. Even so, I find her observations credible not so much because I find her credible, but because so much of what she says is visible when he talks on television and in his tweets.

More revealing is how she portrays the Trump family and the Pence contingent. She said that we should think twice about impeaching Trump because three weeks later we’d be dying to get him back. She thinks Ivanka encourages her father’s handsy lust for her and uses it to her advantage. I’m not so sure about that. Ivanka seemed pretty creeped out to me when Trump went after her in public during the campaign and bragged how he would date her if she weren’t his daughter. She has some nice things to say about Melania, though, perhaps surprisingly.

No one is all that interested in this book as a literary piece, but I must state that it is badly written and apparently totally unedited. It’s replete with grammar and spelling errors. I won’t bother to list those, but more disturbing are some of the logical brain benders that made it to print. For example: “Walking into the briefing was like coming through the tunnel of a visiting team’s field.” Huh? She said this about entering a meeting of people she thought were hostile to her. The visiting team’s field would be empty because, well, it’s visiting, not at home that day. If she meant she was in the position of a visiting team, she would be walking into the home team’s field. She repeatedly mentioned that she was like a guardrail protecting the world from the “Trump train.” A train with a guardrail? I don’t think it would do much good stopping a train, and she didn’t.

With all that said, it’s probably a viable and useful insider look at today’s White House and will end up as a source for future historians, so I recommend reading it if you care about what’s happening there.

View all my reviews

Cipher analysis – the Condex

A few years ago I developed a statistical test called the Normor test that measured how closely the letter frequencies of a cipher resembled normal letter frequency. This turned out to be quite useful as a diagnostic tool for identifying the type of an unknown cipher. One shortcoming of that test is that it does not distinguish between transposition types. They all have the same Normor score as the plaintext. It occurred to me that something similar could be devised that measures contact data to see how closely that data looks like normal contacts. This might possibly be useful in distinguishing between transposition types or even other types.

First I had to write a program that tabulated contact data in a usable form. This proved to be a bit of a programming challenge for me, but I succeeded in writing a program that put the data in a form similar to the chart appearing on page 220 of Elementary Cryptanalysis (Elcy) by Helen Fouché Gaines. I used the program to produce the following chart from dozens of novels, speeches, and other English-language materials downloaded from Gutenberg.org.

MRSHE A NTRLS
DSOEA B EOLUA
SNAEI C OAHET
LIAEN D EIOAT
VLTRH E RSNAD
NSIEO F OITRA
UEAIN G EHAOR
GWSCT H EAIOT
SHLRT I NSTCL
YTSEN J UOAEI
ONRAC K EISNA
OIELA L EILAO
UIAEO M EAOIP
UEOAI N TDGEO
HSNRT O NRUFM
MAOSE P EORAL
ANISE Q UAIEH
TUAOE R EOIAS
URAIE S TAEIO
EIANS T HOIEA
RTBSO U SRTNL
ROIAE V EIAOY
DSTEO W HIAEO
SAOIE X TPIEA
ETARL Y OSTAI
OEZAI Z EAIZO

This differs slightly from the Elcy chart in that I limit the contacts to five on each side, but the data is much more inclusive since it is based on much more data. Use the central letter and look outward to see the letters that most frequently immediately precede (left side) or follow (right side) that central letter. For example, the letter that most often contacts Y on the left is L. The second most frequent one is R. Similarly on the right the most frequent contact is O, then S.

I use this table as my normal English standard. The program was then run on some sample ciphers. Since they are typically too short to fill both sides of the table, I do that with periods. Here’s a columnar cipher and the resulting table:

srwhogteratwiabrndhgpiainishewslalleuniiobysonoooteiftaosslhnaietnesemtnkmfosutiaetasthoihsrtitafuhrenoeeteegfesooshahttrenpdtlvhidurrbsnossnoeseqarebdmgssmetef

nlhti a ibefh
.roea b drsy.
….. c …..
.pnib d hmtu.
soert e tsenb
migea f eotu.
.omhe g fpst.
lidas h oaegi
hetna i adefh
….. j …..
….n k m….
.tlas l aehlv
.sked m efgt.
ihtse n oiade
ashon o soebg
…ng p di…
….e q a….
hebas r eabnr
baseo s sehln
gfdae t eainh
.sfed u hnrt.
….l v h….
..tre w his..
….. x …..
….b y s….
….. z …..

When two letters share the same frequency they are listed in alphabetical order from inside out. This contact chart could be useful solving many ciphers such as cryptograms by hand, but my aim was to measure how much this set of contacts matches the standard above. After some experimentation I found the best way to do this was to go row by row and take each character in this target ciphertext that appears to the left or right of the central letter and take the difference between its position in this lower chart and its position on the same side of the same row in the normal chart and keep a running total.  For example, for row B, the letter A is the most frequent left contact in both charts so the difference in positions is 0. For the right side, the D is most frequent in the cipher but doesn’t appear in the normal, so I add 5 for each such instance. For the K row, N is in position 1 in the cipher, but 4 in the normal chart, so the difference of 3 is added. When all 26 rows are totaled, I divide by the total number of letters appearing on the right and left sides of the cipher (ignoring periods) to arrive at an average position difference. I call this number the Condex for Contact Index. If the cipher contacts exactly matched the normal chart, the total (and average) would be 0. If none of them appeared at all in the normal list, it would be 5. In short, the higher the score, the less normal.

I found that English plaintext averaged in the low 2 range, i.e. 2.0 to 2.25. I tested paragraphs of some novels and the highest average score was 2.487, with a single high of 3.06 and a low of 1.74. My file of ACA solutions averaged higher, 2.79, but bear in mind that it contains very non-standard constructions like the Patristocrat specials and Playfair solutions with X’s between the doubled letters. When I tested several transposition cipher types (testing hundreds to thousands of each type) I found they averaged in the mid- to high 3’s. In order from low to high they were Amsco, Myszkowski, Columnar, and Swagman. The average score and ranges of the latter three were nearly identical, but the Amsco was noticeably lower, which makes sense since the typical Amsco ciphertext consists of about 2/3 normal digraphs. It averaged 3.45. Amscos were the only ciphers I tested that had scores below 3, going as low as 2.8. The lowest among the others was one Swagman con at 3.15. Thus the Condex could be helpful in identifying an unknown Amsco. However, I must note that there are other easier ways to do that such as counting common digraphs.

For non-transposition types the scores were much higher, both the average scores and the maximum and minimum scores. I tested the following types: Bifid, Two-Square, Foursquare, Fractionated Morse, Quagmire,  Bazeries, and Vigenere. I used Bion’s 2-square/4-square data for those types and generated my own for the others. The differences in ranges of scores were so slight as to be meaningless. The averages ranged from 4.12 to 4.29. The Two-Square had the biggest variation and some of the lowest ones dipped down into the mid-3’s. The Condex might be useful in distinguishing between transposition and substitution or fractionation types, but that, too, is more easily and accurately done with the Normor or other tests.

The algorithm is too computation-heavy to be used in any iterative solving process like hill-climbing and I don’t see how it would help there, anyway. Although I don’t see any future as a type diagnostic tool for the Condex, the tool is at least useful for some hand-solving and might prove useful for tabulating data for foreign languages. Anyone who wants to experiment with it, contact me and I’ll provide you with my Windows executable program. There’s a contact link in the top menu.

These results are valid only for text lengths in the typical range for ACA ciphers. I used a minimum of at least 100 letters for my testing and anything below that becomes almost random, even for plaintext. The maximum length was probably around 300 letters. For very large data samples of English, for example, the score will drop virtually to zero.

More Google NGram tales

Once again I am posting stories concocted entirely by the Google NGram site. I started each sentence with three or four words and an asterisk, as indicated by the underlines, and Google provided the next word by listing whatever most often came next following those exact words in the millions of books that it has scanned. I then repeated, dropping the first word and using the new word until a full sentence was achieved. Thus each word is a function of the preceding three or four words. Voila!

The President might have dissolved it by withdrawing the army and navy. His wife never knew whether he was in the habit of doing so. Congress reacted by passing the gas through a solution of potassium iodide and starch. Then the Supreme Court ruled that the state had a duty to perform in the future as a result.

Her beauty was not of the same kind of thing as a matter of fact. It was even more important than the other two groups. When he saw her, he was so sorry for her. That’s why she‘s so upset about the death of the body.

Something in the Water by Catherine Steadman

Something in the WaterSomething in the Water by Catherine Steadman
My rating: 4 of 5 stars

I normally don’t like books read by the author nor books written by actors or other celebrities, but this book is an exception to both rules. The author is an actress, a very accomplished one from shows like Downton Abbey, so her voice acting on the audiobook was superb. Maybe I’m just a sucker for a posh British female accent but I loved hearing her read. The story is a ripping good thriller, too. I wouldn’t call it a mystery. It doesn’t begin with a murder, or at least not an obvious one, but it opens with Erin, our heroine, digging a grave. After that and returning to the actual start of it all, it’s in straight chronological order, which I much appreciated. Erin and her husband honeymoon in Bora Bora where they find something in the water while out scuba diving, something that changes their lives. It’s valuable, but maybe too valuable – something wanted by some very bad people. Are they safe? Do they keep it? Read the book to find out.

There was one stylistic quirk that bothered me. Erin narrates the story in the first person and is continually second guessing herself. “Now we’re safe. We are … aren’t we?” “I’m an honest person. I am. Right? Or am I?” That sort of thing. I think the author was trying to throw some suspense into everything and it only became an irritating affectation, but only a minor one. It’s a worthy read.

View all my reviews

Our Ignorant Newsies – Axe to Pick

My wife caught this one while listening to the radio. I don’t know who the commentator was, but my wife usually listens to PBS. Someone reportedly had “an axe to pick” with someone else. I suppose that’s much like having a bone to grind, but it sounds a bit more violent. They both sound pretty violent when you think about it – not very PBS-like. They should take a pickaxe to both phrases.

Ghost Fleet by P.W. Singer and August Cole

Ghost Fleet: A Novel of the Next World WarGhost Fleet: A Novel of the Next World War by P.W. Singer
My rating: 3 of 5 stars

This naval war novel is very much in the thematic style of Tom Clancy. The title refers to an imagined scenario where most of the digital weapons the U.S. has, such as GPS satellites and chip-dependent aircraft, have been neutralized by a Chinese malware package. Chinese and Russians are allied against America. War ensues and the Chinese “Directorate” dominates at first. America thus turns to its older fleet of warships and planes, the so-called ghost fleet or mothball navy floating uselessly now in real life in Suisun Bay, to fight back.

It’s a clever scenario. The writing, however, doesn’t live up to the premise. The first 300 pages are a slog. I had trouble keeping everybody straight. There are too many characters and settings. Bad Chinese and good Chinese-Americans. Good Russians and bad Russians. Two characters, father and son, are named Simmons which causes additional confusion. The scenes and settings are very short jumping all over the place. Zillions of military acronyms and alphabet soup weapons system names are bandied about endlessly.

It takes way too long to get to the actual battle action. Despite this, the final 100 pages or so are pretty exciting and make it worth the three stars. I was surprised at the political correctness for such a macho-themed book. Half the military personnel are women (often with male-sounding names or nicknames, which only added to the confusion). A gay officer is even thrown in for a cameo. I gave up on the audiobook the first time I tried this one, and would have given up on it entirely, but since it was a selection for my book club I forced myself to get the print book and read it through. In the end, it was okay but I can’t really recommend it.

View all my reviews

Paper Ghosts by Julia Heaberlin

Paper GhostsPaper Ghosts by Julia Heaberlin
My rating: 3 of 5 stars

I was leery of this one at first since it was supposed to be about serial killings. I expected some gore and sadism. It was indeed about serial killings, but it did not describe them in detail or with much gore. The story is told by a young woman using different names whose sister Rachel was killed. Our protagonist believes the killer is a mentally disturbed man who used to be a famous photographer and who had once taken photos of her family and other murdered girls, a man who was once tried and acquitted of the murder of one of them. She pegs him for three unsolved murders and sets about on a long-term plan to lure him from the assisted living home on the pretense she is his daughter (learned only through a DNA match). The book is the tale of their journey together the the rural south. The book held my interest and the author captures the feel of the country setting well.

View all my reviews