Econometric Models

This post really annoyed me (emphasis mine):

The ARRA, the fiscal stimulus act passed last year, gave the Council of Economic Advisers an impossible job: measuring how many jobs the act created. Here is the CEA's latest attempt. As far as I can tell, there are two kinds of evidence here.

First, there are model simulations. That is, the CEA took a conventional Keynesian-style macroeconomic model and used those set of equations to estimate the effect the stimulus should have had. Essentially, the model offers an estimate of the policy's effect, conditional on the model being a correct description of the world. But notice that this exercise is not really a measurement based on what actually occurred. Rather, the exercise is premised on the belief that the model is true, so no matter how bad the economy got, the inference is that it would have been even worse without the stimulus. Why? Because that is what the model says. The validity of the model itself is never questioned. (Moreover, the fact that other organizations simulating similar models come to similar conclusions is no evidence about the validity of the model's simulations. It only tells you the CEA staff did not commit egregious programming errors when running their computer simulations.)

This criticism is vacuous in the following sense:
1. It is valid not just for this model, but any econometric model.
2. It is valid also for any DSGE models that assume exogenous 'deep' parameters.
3. It is essentially valid for any econometric model that uses instrumental variables or 'fancy' causal type analysis. (Can any one say valid instruments?)
4. Any model is only as good as its assumptions.

The following is just an off-the-cuff remark and I'll probably get into trouble for this: No one really is convinced by any econometric or statistical evidence especially if it is a one-off study. We are only convinced if we already believe in the first place and in this case, the author has already discounted the effects of ARRA (before any evidence was even presented) and therefore dismisses this evidence as unconvincing.

Even if I were agnostic about the effects of the stimulus, this one study would really do nothing to convince me that it is effective. Different models, using different assumptions (robustness is what some may call it) really is the key to trying to persuade. After all, models (econometric or otherwise) really are rhetoric in disguise.