Are DSGE Models "Sloppy"?

(Published by JP Bouchaud | December 2022)

I have long been ranting about some of the economists' sacred cows, like the Black-Scholes equation or the Dynamic Stochastic General Equilibrium (DSGE) model. So I felt an undeniable Schadenfreude when I read the recent paper of McDonald & Shalizi, where they show that

“…even if the (Smets-Wouters) DSGE model was completely correct about the structure of the economy, and it was given access to centuries of stationary data, it would predict very badly, and many parameters would remain very poorly estimated”,

a conclusion jeopardizing hundreds of papers in the field and shedding strong doubts on the usefulness of the models routinely used by most Central Banks to guide their monetary policy.
 
Well, it turns out that the McDonald-Shalizi paper is perhaps not as scathing as it looks. As explained in detail in this blog, DSGE specialists have in fact long been aware of (some of) the pitfalls encountered when trying to calibrate large DSGE models to data. Several heuristic tricks have been proposed to guide the calibration towards “reasonable’’ values, so that the parameters of a DSGE model can be correctly recovered using its own simulation output – a minimum requirement indeed.
 
However, as with many models used in different areas of science, DSGE models turn out to be “sloppy”, as Jim Sethna and associates like to call them.

In their own words:

“Complex models in physics, biology, economics, and engineering are often sloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions.’’

In other words, it is in general both hopeless and useless to try to calibrate *all* the parameters of a large model.  Rather, one can and should prune large model down and only retain their “stiff’’ (as opposed to sloppy) backbone. What is cool is that this can be done in a principled way, in particular using information geometry and geodesic methods, see their fascinating paper here.
 
Coming back to the DSGE saga, Karl Naumann-Woleske, Max Sina Knicker, Michael Benzaquen and myself have shown that most large macroeconomic models to date, be they Agent Based Models or DSGE models, are indeed sloppy, see this paper of ours.
 
The basic tool here is the Hessian matrix of the loss function when parameters are slightly changed (see figure for the eigenvalues of the Hessian matrix; e) is DSGE, note the log scale).  We have shown how this tool helps navigating efficiently in a large dimensional space of parameters. But an even more exciting avenue is to follow Sethna’s suggestion and build educated *reduced form* models that would be much more robust and easier to calibrate.