Validating CGE models

MR pointed to the following post on CGE modeling:
"As far as I know, there has never been a rigorous ex post evaluation of CGE models in practice, one that compares predicted to actual outcomes. Based on performance, is there any evidence that such models add value—that their predictions are any better than those derived from macro or sector-specific models, or even a random walk?"

This is not new and in fact this challenge has been issued and taken up by Tim Kehoe in "An evaluation of the performance of applied general equilibrium models of the impact of NAFTA"
This paper evaluates the performances of three of the most prominent multisectoral static applied general equilibrium models used to predict the impact of the North American Free Trade Agreement. These models drastically underestimated the impact of NAFTA on North American trade. Furthermore, the models failed to capture much of the relative impacts on different sectors. Ex-post performance evaluations of applied GE models are essential if policymakers are to have confidence in the results produced by these models. Such valuations also help make applied GE analysis a scientific discipline in which there are well-defined puzzles with clear successes and failures for competing theories. Analyzing sectoral trade data indicates the need for a new theoretical mechanism that generates large increases in trade in product categories with little or no previous trade. To capture changes in macroeconomic aggregates, the models need to be able to capture changes in productivity.

The main problem with validation is the ceteris paribus approach in CGE modeling and the fact that a lof of them are static so the time of transition from one state to another is left unaddressed. For instance, if the CGE model predicts that with a change of taxes from 10 to 11 percent will reduce consumption by 0.2 points we need to confirm this by taking the model to the data.

We can start with the current data but when should we look again after the 1 percent increase in taxes is implemented? Do we look at consumption data at the time the 1 percent increase, 1 quarter after or 1 year after? Most static CGE models are silent on this issue. In addition, there are possibly multiple shocks to the economy between two points in time so even if consumption did fall by 0.2 points can we really attribute all of this fall to the 1 percent increase in taxes?