The failure of models

Not financial models this time but weather models! Via Capital Weather Gang:
... what may have turned out to be the blizzard of 2009 ultimately existed solely in computer model simulations of the weather. Despite their best efforts, the models with impersonal acronyms like "GFS" and "ECMWF" could not force the storm scenario they had predicted to come to fruition. A dusting of snow fell around Washington, and a few inches in New York and Boston, but that was far from the one to two feet that was projected at one point.

The non-storm event provided an interesting case study in how communicators of weather information should balance the need to attract eyeballs, in the form of viewers and readers, with the uncertainties inherent in forecasting that can make even the best forecast go bust. Remarkably, despite an early consensus of the computer models that a major storm was likely on Groundhog Day (that consensus had crumbled by last weekend), most media outlets -- including the Capital Weather Gang -- limited their hype to a Category 2 situation rather than a Cat 5 "hype-cane."

Although there were probably others who leaned too heavily towards hyping the storm, the most notable example of a media organization that balanced the ratings/uncertainty equation in favor of hyping the storm beyond what was meteorologically justified was AccuWeather Inc., the Pennsylvania-based private forecasting company that has a reputation in the weather forecasting industry as frequently being bullish on predicted snowfall amounts.

Update from CWG :
At times toward the end of last week there was a consensus among the computer models meteorologists use to help predict the weather, which go by acronyms such as "GFS," "ECMWF" and "UKMET," that the AccuWeather forecast could come true. But, there were also times when these models fluctuated wildly, showing a big snowstorm on one run and virtually no snow at all on a run six or 12 hours later (most models are run every six or 12 hours).

... Nevertheless, it must be emphasized that up until late Saturday and early Sunday there was a very realistic possibility of AccuWeather hitting a home run. -- sometimes even blind squirrels find an acorn. Had the storm occurred as AccuWeather had predicted, you'd likely never hear the end of AccuWeather claims about how wonderfully skillful it had been -- rather than acknowledge the role of old fashion good luck.

... Generating a more complete picture of the full range of possible forecasts is the basis for ensemble forecasting, where a single model is run up to 50 times, each time with a set of slightly different initial conditions (to account for possible inaccuracies in the measured initial conditions). Each of these runs is known as an ensemble member, and together the ensemble members provide a reasonably good portrayal of possible outcomes (e.g., storm track and intensity) and their relative likelihood. The more members that show a given scenario -- for example, a storm hugging the coast and producing significant snows, or instead a storm tracking too far off the coast to produce meaningful precipitation -- the greater the chance that scenario will pan out (at least in theory).

Some lessons here for economic models as well?