Good models and Verisimilitude
(Published by JP Bouchaud | October 2024)
I just finished reading for the second time my friend Doyne Farmer's book "Making Sense of Chaos". Lots to take home, in particular about how to build "good models". Quote from his book:
Milton Friedman argued that models should be evaluated *solely* on how good their predictions are, rather than on the plausibility of their assumptions. Even if the assumptions of a model are far-fetched, he wrote, the economy may behave 'as-if' they are true.
[...]
We should be very suspicious of models that require implausible 'as-if' arguments. Instead, I believe we should follow what I call the Principle of Verisimilitude: Models should fit the facts and their assumptions should be plausible. Assumptions that seem wrong from the outset are more likely to lead to false conclusions than plausible assumptions. We need to replace 'as-if' reasoning with 'as-is' reasoning.
I cannot agree more. There are so many models based on a priori absurd assumptions. One example is the slew of "copulas" invented to model multivariate financial time series. As we argued with Rémy Chicheportiche in this paper.
"The search for a theoretical description of multivariate dependences has led to the explosion of the literature on copulas. Several families of copulas have been proposed: Elliptical, Archimedean (Clayton, Franck, and others), Vine, Liouville, etc. Unfortunately these copulas are often theoretical figments with no financial interpretation, and for that reason alone should be considered with suspicion. (These alternative copulas also fail to reproduce the empirical dependences of stock returns). We advocate the need for constructing meaningful copulas, based on intuition and plausibility."
The whole idea is, in a sense, an extension of Bayes reasoning: we should attribute, at least qualitatively, low prior probabilities to models that rely on implausible assumptions. Being able to judge the plausibility of model assumptions requires training much beyond mathematical skills, but a slow, patient back and forth between models and reality- something physicists are exposed to early on in their curriculum.
Have you tried the Francis Ford Copula ? :) :)