The limitations of mathematical modeling

Escape from Model Land: How Mathematical Models Can Lead Us Astray and What We Can Do About It

by Erica Thompson, Basic Books, 2022

The single most well-known climate forecast in British historical past can also be one of many worst. In October 1987, a meteorologist working for the British Broadcasting Corporation reassured a involved viewer that rumors of an approaching hurricane had been unfounded. Hours later, 22 individuals had been killed and billions of kilos of injury performed by extremely uncommon hurricane-force winds. Although the faulty forecast owed to a scarcity of knowledge in elements of the North Atlantic, it was the meteorologist, Michael Fish, who grew to become a synonym for flawed prediction fashions. The work of everybody who makes use of such mathematical fashions to provide explanations of how sophisticated issues work is the topic of a brand new ebook by Erica Thompson, a tutorial on the London School of Economics. Her rivalry is that too many people have change into ensconced in a cushty however finally unhelpful place, which she dubs Model Land.

Thompson believes Model Land is a superb place for theorists—economists, climatologists, financiers, political scientists—as a result of fashions are totally controllable. Experimenters can set the parameters, run their exams, and write with confidence about their outcomes. There are not any messy or uncomplicated components. “Whole careers may be spent in Model Land,” Thompson writes, “doing troublesome and thrilling issues.” Except these items aren’t actual. Or reasonably, they don’t apply to the true world. It is that this delusion that has led governments and companies which might be unquestioning of mannequin outcomes into hassle—and prompted Thompson to write down her redress, Escape from Model Land.

In the late Nineteen Forties, one in every of Thompson’s predecessors at LSE, an undergraduate named Bill Phillips, and his colleague Walter Newlyn, constructed a bodily mannequin of the UK financial system out of water tanks, pipes, and pumps to reveal how one thing very sophisticated works in a simplified, visible means. The water represented the stream of cash by way of the financial system and was held in tanks, representing banks and the federal government, whereas the pump represented taxation. The intention, as Thompson interprets, was to set the pumps and valves at a stage “that permits for a closed loop that forestalls all the water ending up in a single place and the opposite tanks working dry.”

But to Thompson, Phillips’s work is a product of Model Land. “The solely means that the Phillips–Newlyn machine can characterize financial failure, for instance, is by the running-dry of the taxation pumps; there isn’t a idea of political failure by…failing to supply satisfactory public companies. And the one success is a continued stream of cash.” In different phrases, the land of Phillips’s mannequin is a artistic and admirable approximation of how actual economies work, nevertheless it has edges and limits to its scope that don’t apply to the true world.

And after we let these fashions dictate habits in the true world, we flirt with catastrophe. Thompson writes a superb part on monetary crises as an example her level. The traditional mistake made by banks, hedge funds, and different traders is “assuming the information we’ve are related for the longer term we anticipate.” During occasions of stability, this method may be worthwhile. But when occasions that fashions imagine are most unlikely all of the sudden materialize, because the collapse of Southeast Asian currencies did in 1997–98 or the unravelling of the US mortgage market a decade later, model-guided traders may be caught out.

Thompson believes these failures are sometimes owing to misaligned incentives: “Those who appropriately estimate important tail dangers [i.e., deviations from the normal distribution in a statistical model] will not be acknowledged or rewarded for doing so. Before the occasion, tail dangers are unknown anyway if they’ll solely be estimated from previous information,” and “after the occasion, there are different issues to fret about.” In brief, it was in traders’ curiosity to design a mannequin that characterised unlikely dangers as infinitesimally so, and regulators weren’t paying consideration.

So why ought to we trouble with fashions in any respect? Occasionally, Thompson believes, they do get it proper. Her most well-liked instance issues analysis by two chemists, F. Sherwood Rowland and Mario Molina, who within the Nineteen Seventies modeled the potential impression on the ozone layer of the continued launch of chlorofluorocarbons, or CFCs. Within 15 years of their analysis, a world settlement, the Montreal Protocol, had been signed to restrict CFC use, and it’s now doable that the ozone layer might get better to its 1980 stage by 2050. “The acceptability of the mannequin was a operate of the comparatively easy context and the low prices of motion,” Thompson explains, earlier than warning, “that is in direct distinction with the scenario for local weather change.”

In the tip, Thompson comes again to consultants. Michael Fish interpreted his information appropriately, however the information itself was not enough. The actual mistake made by the Met Office, the UK’s nationwide climate service, was placing an excessive amount of religion in its mannequin. She recounts the Challenger catastrophe in 1986. Previous missions had revealed a number of faults within the area shuttle’s O-rings, which sealed its rocket boosters. Some engineers had calculated that the chance of a significant catastrophe was excessive. Others noticed it in a different way: the truth that Challenger had been in a position to full the earlier flights offered an information set that underlined its power. “On the face of it, and close to the information,” Thompson argues, “both situation is possible.” It is just when the modeling is supplemented with professional judgment that we stand an opportunity of escaping from Model Land and discovering ourselves with info that may be utilized in the true world.