Can we really predict the unpredictable using mathematics and data ?

© Markus Spiske/Pexels

The dream of predicting the future has always haunted the human mind. From the oracles of antiquity to contemporary algorithms, this quest spans the centuries, taking on ever-changing forms. Today Today, at the University of Oxford, physicist and economist J. Doyne Farmer proposes a revolutionary approach to prediction and explains it in his latest book, Making Sense of Chaos: A Better Economics for a Better World.

The latter is rooted in the computing power of modern systems and chaos theory, which explores how deterministic systems, subject to precise rules, can generate unpredictable and complex behaviors due to extreme sensitivity to initial conditions. Farmer's atypical career illustrates a deep conviction: the apparent unpredictability of certain phenomena often masks regularities that science can decipher. Paul Rand gave him the floor for the 147th episode of the American podcast Big Brain; this is what came out of it.

J. Doyne Farmer, a pioneering scientist in quantitative trading. © Magnus Manske/Wikipedia

From chance to physical laws: the birth of an unprecedented method

Farmer's first steps into the world of prediction began in front of a casino roulette wheel in the 1970s in Las Vegas. Where ordinary mortals see only a game of chance, the physicist detects a mechanical system governed by Newton's laws. “&nbsp ;Wait a minute, it’s just a marble rolling on a circular track with a rotor in the center, spinning in the opposite direction. These are simple physical systems, so as physicists, we should be able to predict their behavior ,” Farmer explains.

By applying these fundamental laws and precisely measuring the physical forces at play, his team developed a mathematical model that could predict the ball’s trajectory. To exploit this discovery, they created the first concealable wearable computer—a remarkable innovation for its time. The device, hidden under their clothing, received data via switches activated by their toes.

When the ball passes a reference point, the operator sends a signal. The computer then calculates the ball's speed and predicts its probable arrival zone. A second player can then place their bets accordingly. This scientific method allowed them to gain a huge statistical advantage over the casino. “We beat the casino by a good margin, about 20%”, Farmer explains.

This experience, beyond its anecdotal aspect, lays the foundations for a deeper reflection on the nature of chance and predictability. It demonstrates that an apparently random phenomenon can become predictable when we understand the physical mechanisms that govern it and have the appropriate mathematical tools to model them. A lesson that Farmer would later successfully apply to a completely different sector.

Complexity as a key to reading financial markets

Building on this success, Farmer transposed his methods to the financial markets. In the 1990s, he founded a company called Prediction Company, which would be the equivalent of a jolt in the world of finance. It would be one of the first companies to use computers to perform stock market transactions fully automatically, following mathematical and statistical models.

Unlike casinos, where winnings quickly attract the hostile attention of establishments, the financial markets offered Farmer an ideal testing ground. “As soon as we started winning well, the casino put pressure on us, so we disappeared. We enjoyed ourselves, but we can't say that we made a fortune ” emphasizes Farmer.

200% Deposit Bonus up to €3,000 180% First Deposit Bonus up to $20,000

In partnership with UBS, the company develops sophisticated trading strategies, based on the analysis of complex systems. The results exceed all expectations: over 28 years of activity, the company has made profits for 27 years. Even more impressive, the return/risk ratio of their investments is six times higher than that of the market. A success that owes nothing to chance. Where classical financial theories postulate the efficiency of markets and the perfect rationality of investors, Farmer adopts a radically different approach.

His team then models markets as complex systems, comparable to the natural phenomena he studied as a physicist. Price fluctuations are no longer seen as a series of random numbers, but as the result of multiple interactions between actors with imperfect behavior. This approach makes it possible to capture subtle dynamics: cascade effects, bubble formations, panic movements. By integrating investors' cognitive biases and their real strategies – often far removed from pure rationality – their models manage to anticipate market movements invisible to traditional approaches.

The silent revolution of ” complexity economics “

After these successes, Farmer is now tackling an even more ambitious challenge: modeling the economy as a whole. His new company, Macrocosm, is developing an economic navigation tool that visualizes the complex interdependencies between actors and simulates the impact of different decisions. Farmer compares it to a well-known tool, Google Maps: ” My ultimate dream is to revolutionize economic decision-making like Google Maps has transformed traffic management “.

Baptized ” complexity economics », this approach radically breaks with traditional economic models. Instead of relying on mathematical equations assuming perfectly rational agents, Farmer's strategy simulates millions of economic actors making imperfect decisions, based on simple rules of thumb. A business manager does not calculate an optimal strategy, he imitates his successful competitors. A consumer does not maximize an abstract utility function, he follows habits and heuristics, etc.

The effectiveness of this method was particularly evident during the COVID-19 crisis. Farmer’s models predicted a 21.5% fall in UK GDP in the second quarter of 2020 – the reality was 22.1%. A remarkable accuracy that has gradually convinced major institutions. Several central banks, including those of Canada and Italy, are now integrating these tools into their decision-making. The potential applications are vast: from anticipating financial crises to optimizing energy transition policies, to studying inequalities. A new way of thinking about economics is emerging, closer to the complex reality it seeks to understand.

This new science of complexity could therefore make our predictions more reliable in many areas. Not by predicting the future with absolute certainty – an ambition that will always remain illusory – but by identifying the patterns and dynamics that structure complex systems.

• Doyne Farmer has used chaos mathematics to make complex phenomena, from casino roulette to financial markets, predictable.
• His models, based on real human interactions and not perfect rationality, capture often invisible dynamics.
• Today, he aims to completely transform economic decision-making with his company Macrocosm.

📍 To not miss any Presse-citron news, follow us on Google News and WhatsApp.

[ ]