Comparing economic research to meteorological research

From Storm World: Hurricanes, Politics, and the Battle Over Global Warming by Chris Mooney.

While reading this it is hard not to draw parallels between economic modeling and weather forecasting - the balance between models and data and the need to simplify the models:

On comparing models with data: Theoretically based predictions, however, don't hit you in the gut like hard data. (p. 8)

[This] seems to be a rule in our science: progress is impeded by want of meteorological knowledge on the part of theoreticians and by a too poor mathematical training of weather-men.
-Swedish meteorologist Tor Bergeron, "Methods in Scientific Weather Analysis and Forecasting", 1959 (Introduction to Part I)

... in the absence of a complete theoretical understanding, models themselves can become the source of experiments. Vary the equations or other aspects of the model, plug the data back in, and see how much closer to reality you can get. But from the perspective of some scientific empricists, this seemed an ungrounded and even suspicious way of doing things. "Tuning" the models to make them line up better with observations sounded like rigging the game. (p. 50)

For an analysis of season length, the record breaking 2005 Atlantic hurricane year provided only one data point and could not in itself justify broader conclusions. (p. 179)

Yet data alone, without a physical understanding of what's happening, can also blind and mislead. Correlations don't prove causation. Sahel rainfall can change in lockstep with Atlantic hurricanes without causing changes in that activity. ... That's why any healthy science will inevitably balance both theoretical and empirical approaches. ... No such model result should be considered an unerring prediction; instead climate models are perhaps most useful when employed to test hypotheses thtat scientists come up with about how the real world works, and what is likely to happen if various natural or human influences occur in the future. Whatever the inevitable shortcomings of a given model, if it contains the relevant phyiscal processes and gives the expected result, the hypothesis has at least been confirmed within the constraints of that particular model. (pp. 268-269)

On simplifying features in models and fads in the field:

CISK (conditional instability of the second kind) had a number of key problems, many of which sprang from the attempt, so characteristic of Charney, to strip hurricanes down to mathematical essentials rather than study them in their full-blownn reality. (p. 52)

"There are fashions in science, and that was a fashion," recalls University of Oklahoma meteorologist Doug Lilly, a skeptic of CISK who supported a "heat engine" revival in the 1980s. (p. 52)

From the 1960s until the 1980s, when it began to fall out of favor, CISK thus distracted attentiona away from the concept of hurricanes as ocean-driven "heat engines." In fairness, the theory also prompted a great deal of thinking - wrong ideas can be productive in that way. But CISK has also been characterized as a "setback" for the field ... By then, however, CISK was well on its way to becoming yet another dominant paradigm. (p. 53)

... [William] Gray replied, "and the trouble with that is they don't know how the atmosphere ticks. They're modelers. The're people that make assumptions that are not valid, and they believe them." (p. 174)

Judith Curry on interactions with media, politics and science which were excised from her article:

The prevailing views on the topic of hurricanes and global change differ considerably between hurricane forecasters and climate researchers. The consensus view of hurricane forecasters is to attribute the warming in the North Atlantic and the associated in increase in hurricane frequency and intensity to natural variability. The consensus view of climate researchers is to attribute the warming, particularly since 1970, to have a substantial component associated with greenhouse warming. These discrepancies can be understood at least in part by clarifying the source of these differing perspectives. The hurricane forecaster focuses on predicting the path and intensity of land falling hurricanes, and also makes seasonal forecasts. They work on verifying their forecasts, and they are also experts on hurricane data. On the other hand, the climate researcher does not focus on forecasting but rather applies the scientific method to understanding the underlying physical processes and causes of climate variability. The climate researcher has expertise on climate data records and statistical methods. (pp. 237-238)

The rest of the excised material is the following paragraph from RealClimate.org:

The richness of the meteorological community, including both scientific researchers and the operational forecasting community, provides the community with both benefits and challenges. As a result of the utility of operational forecasting and the utility of the reanalysis products, some sloppy practices have crept into the meteorological research community in terms of careful assessment of the errors of data sets and hypothesis testing. The public views the meteorological community in a monolithic way and seems prepared to accept the opinions of TV weather forecasters on issues such as global warming, in spite of the fact that this community has most often no expertise on this topic. A dichotomy has developed in the U.S. between the operational forecasting community and the meteorological research community, a dichotomy that does not exist in Europe. Some of the most challenging scientific issues that are also of the highest policy relevance are at the interface of climate change and weather extremes. The operational forecasting communities and the research communities need to work together on these issues, and NOAA and the AMS can play a major role in facilitating this collaboration. We must make every effort to avoid institutionalized scientific bias in our community, whereby an organization or group of scientists or other professionals discount what is not known by them personally or collectively, or a group of scientists becomes protective of a scientific research area as being their ‘turf’. The end result of this debate is likely to be that this public fragmentation of the meteorological community has generally lessened the possibility for this community to influence policy.

[These are comments by Judith Curry on Real Climate].

While the subject matter of atmospheric science and forecast meteorology shares much common ground, forecast meteorologists operate in more of an engineering environment relative to the research branch of atmospheric science (I value a good weather forecast as much as anyone!). Bill Gray is an interesting ‘hybridâ’ in that while he worked for decades in the university environment and publishes frequently in the scientific literature, his heritage and mode of thinking seems to be more in line with the meteorological forecasting community (Bill Gray’s lengthy interview with Joel Achenbach would seem to support this characterization).

Meteorologists with a B.S. degree (note many TV meteorologists do not even have this credential) would rarely take a course in climate and global change; this course is not even listed on the NOAA/NWS or AMS certification guidelines for meteorologists. This lack of knowledge even trickles up to the Ph.D. level in meteorology, where I suspect many Ph.D. meteorologists have never taken a course in climate and global change. Further, the entire meteorological education is focused on forecasting: understanding short-term weather patterns, looking for analogues, etc. and the ‘experience’ of weather forecasters is actually important here in being able to call up ‘analogues’ of past disturbances or seasons. Bill Gray certainly has more experience than anyone in this regard in the hurricane world, which is why he has made statements that he is the authority and that ‘we’ are not qualified (again, refer to the Achenbach article), and the length of his experience (50 years) contributes mightily to the support of his views in the hurricane forecasting community. This is vastly different from the research community, whereby a Ph.D. student can legitimately and effectively challenge the research of a Nobel Laureate through the refereed scientific literature. Further, owing to the emphasis on forecasting, this community does not operate in the same way that atmospheric science researchers (outside the forecasting community) operate in terms of hypothesis testing etc., which is why I focused the article in terms of laying out the scientific method, fallacies, etc. (note all of the fallacies came from the hurricane forecasting community via the media; I included specific citations in the 2nd version of the paper, but this was also nixed). Even among Ph.D. meteorologists, global warming is not widely accepted. In addition to the issues previously raised, there has been some ‘resentment’ among the meteorological community about the success (particularly in terms of funding) of the U.S. climate research programs, which they view as coming at the expense of meteorological/weather research (with initiatives such as STORM, the U.S. Weather Research Program, THORPEX receiving orders of magnitude less funding).

This issue re funding for climate science has been mentioned numerous times by Bill Gray in the media. Am I criticizing forecast meteorologists? I highly value their forecasts, and believe that they are for the most part hardworking and honest human beings (although I have my doubts about of few of them in the private sector), and many of them are probably quite brilliant. But something is wrong with the system, and this brings us to NOAA (and to a lesser extent the American Meteorological Society).

Mix all of the above with a ‘political agenda’ that is anti-greenhouse warming with the Undersecretary of NOAA (a political appointee) saying that we do not know what to attribute the recent warming to, then we have a complex situation indeed. NOAA is a large and complex organization, and I don’t envy anyone that is trying to administer all that. But the hurricane and global warming debate has illuminated some glaring problems in my opinion. There is a substantial disconnect between the various branches of NOAA: there are numerous NOAA agencies with substantial expertise in climate change global warming, the National Climatic Data Center (NCDC), the Boulder Labs, and GFDL to name a few. Apparently there is little to no interaction of these agencies with the National Weather Service NWS (although there is apparently some interaction with GFDL). This lack of interaction is to the detriment of both scientific research and the forecasts. The Europeans (notably ECMWF) do not have the dichotomy between forecasting and research, and weather and climate that we see in the U.S., and their forecasts are far better than those in the U.S. (particularly ECMWF).

Now, to the American Meteorological Society, of which I am an active member and have previously served as Councillor. 20 years ago, the AMS was the main professional society for atmospheric scientists: NOAA, university, and private sector with the majority of members from NOAA. As atmospheric sciences broadened as a field and became more interdisciplinary, many of the university types became affiliated primarily with the American Geophysical Union, which is dominated by research scientists. Most of the ‘older’ atmospheric scientists (like me) have maintained a membership in the AMS, but the demographics of the AMS are now such that the membership is approaching 50% private sector. The AMS however maintains an excellent series of scientific journals which score at least as high as the AGU journals in terms of citations/impact. The AMS has struggled in recent years with the conflicts between public sector (NOAA) meteorologists and private sector meteorologists. There is obviously another challenge for the AMS in bridging the broader community of forecast meteorologists with the research community particularly on the topic of climate change.

Particularly on the hurricane and global warming issue, Peter Webster and I are now ‘card carrying’ members of the tropical listserv (Emanuel and Holland are long term members) which is the main venue for communications mostly about hurricanes by operational forecasters with researchers mainly seeming to lurk and occasionally post (note this is not a blog, but a private listserv). I posted the BAMS article on this listserv, so far no one has posted any public responses (although I have received a few personal emails). Obviously a very different response from the realclimate community.

We certainly live in interesting times, and the blogosphere adds a unique element to this, I appreciate the opportunity for a venue to post what I couldn’t publish on the topic.