Taking stock of algorithms’ power

The computer-driven stock trading that produced the “flash crash” of 2010 might seem like the latest tech-driven fad to hit Wall Street, but it has a surprisingly long history.

More than 35 years ago, a Hungarian immigrant to the U.S. named Thomas Peterffy employed 80 programmers to write software that could cull arbitrage profits from financial markets quicker than human traders. That makes Peterffy a pioneer in a pursuit that has spread far beyond Wall Street: using computers to emulate human decision-making, outsmarting or simply outpacing mere mortals in fields once the preserve of experts.

In his new book, “Automate This: How Algorithms Came to Rule Our World,” Christopher Steiner makes a big claim for the mathematical formulas that underpin technology.

“The ability to create algorithms that imitate, better, and eventually replace humans is the paramount skill of the next 100 years,” Steiner says. “As the people who can do this multiply, jobs will disappear, lives will change, and industries will be reborn.”

He may turn out to be right, though the subtitle of his book published by Portfolio sounds decidedly premature. The book’s examples, while intriguing, fail to make the case that algorithms have infiltrated daily life as widely as claimed. In the field of artificial intelligence, it has always paid to take the most expansive machine-replacing-man predictions with a large pinch of salt.

This time, though, two things have made the difference. One is the rapid decline in the costs of computing and communications, making it economical to deploy programs that use heavy-duty number-crunching in many more fields.

Steiner cites a New York Times report that says combing through more than 1 million documents in a legal discovery process can now be done for $100,000 using a computer, compared with the $5 million it would have cost with paralegals on the case.

The other difference is the dawning of “big data” — huge data sets on which to “train” algorithms, refining their ability to find patterns and make deductions from a sea of information.

With a fluid prose style and an eye for interesting characters, Steiner, a former reporter for Forbes magazine and now co-founder of an Internet grocery company, excels in bringing a dry subject to life. The characters he traces help to dramatize one of the central tenets of the book: that techniques invented for solving problems in one field are spilling over into others, accelerating the pace of change.

There is the expert in poker-playing algorithms who repurposes his skills to find donors for kidney transplant patients. Likewise, ideas developed by NASA to identify personality traits in astronauts have been taken up by a call-center automation company. It uses verbal clues to try to categorize callers and prompt workers on how best to deal with them.

Steiner traces the use of algorithms in areas such as music to identify pop tunes with hit potential; or medical diagnostics to analyze test results more efficiently than humans. Yet the examples often sound experimental, and it is hard to assess some of his conclusions.

A rapid and breezy tour of the field, “Automate This” will leave even non-expert readers wanting more detail. Nor does “Automate This” dig far into the social, political and business implications of the sort of algorithmic takeover he predicts. How much freedom, for instance, are algorithms being given to make decisions that affect human beings, and what happens when — as in the flash crash — things go wrong?

By anticipating and influencing behavior, algorithms also open the way for greater political and commercial control. Increasingly, computers are “judging us, routing us, and measuring us.” And the automation of decision processes could be highly disruptive for many white-collar workers.

If Steiner is right, issues such as these will provide ample material for future books.

San Francisco-based Richard Waters is the West Coast managing editor of the Financial Times of London, in which this review first appeared.