Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O’Neil
My rating: 5 of 5 stars
This fascinating and well-written non-fiction book explores how the brokers and manipulators of “big data” affect us all, often in harmful ways. The author is a former math professor, Wall Street quant, and now is a full-time “Data Scientist,” a title she gave herself. She definitely has a great deal of inside knowledge about the users of big data and the algorithms they use to churn through the data and direct their activities. She calls them Weapons of Math Destruction or WMDs. Among the users are banks, credit companies, employers, government agencies, universities, advertisers, search engines, police departments, payday lenders, criminal sentencing courts, and insurance agencies.
She gives an example of a teacher in Washington, D.C. who was fired because a WMD identified her as being in the worst 10% of teachers despite having had glowing reviews in the current year and top scores in previous years. It turns out that the teacher who had taught most of her students the previous year had corrected the standardized tests of her students to make it look like they were performing better than they actually were. The result was that their scores on the test dropped during the next year even though they actually made good progress. The cheater kept her job, while the honest teacher lost hers.
Another surprising area to me was how the U.S. News rankings of colleges and universities has driven up tuition, lowered the quality of faculties, and actually made it harder for some top students to get into a “safety” school. You’ll have to read the book to understand how this happens. Many such tidbits are set forth throughout the book.
The book has a definite political slant to it. The author decries unfairness in general, which I consider apolitical, but then tends to harangue on anything she sees as racial inequality or “targeting the poor.” Many WMDs take into account such things as Zip codes or credit scores, things she considers “proxies” for race. From the viewpoint of a civil libertarian, this is a valid approach, but, as she herself admits, from a business standpoint, some of these WMDs are effective at reducing inefficiencies and increasing profits. Corporations, banks, and even many government agencies are not in the business of fairness or eliminating racial inequality; they’re in the business of business, i.e. making money, or in the case of the government agencies, accomplishing an important task like public safety or building infrastructure at a reasonable cost. One could argue that they have a legal and moral duty, a fiduciary responsibility toward stockholders or taxpayers, to increase those profits or efficiencies. As she also admits, using traditional human judgment alone, without the WMDs, has its own history of unfairness and racial prejudice.
A central theme throughout the book that was not explicitly stated is the failure of nearly all the WMDs to take into account the effect they themselves have on human behavior. Take the mortgage crisis of 2007-2008, for example. The math behind the collateralized debt obligations (CDOs) that packaged sub-prime mortgages with other debts was valid. If borrowers had continued to behave as they had statistically in the past, defaulting at the same rate, the CDOs would have been sound investments. What the lenders and brokers did not factor in was that once the market was created for these securities, lenders and borrowers would both change their behavior, increasing the number of mortgages granted to people who obviously had no way to repay them, thus changing the long-standing statistics on which the WMD was based.
If you want to know how to increase your chances of getting hired or how to get a better college education at a lower cost, this book is worth your time studying. I don’t have any skin in the game, but I found it a very interesting read even so.