Atmospheric Chemistry and Physics Discussions
www.atmos-chem-phys-discuss.net
1680-7367
1680-7375
8
3
2008
10.5194/acpd-8-10873-2008
http://www.atmos-chem-phys-discuss.net/8/10873/2008/
http://www.atmos-chem-phys-discuss.net/8/10873/2008/acpd-8-10873-2008.html
http://www.atmos-chem-phys-discuss.net/8/10873/2008/acpd-8-10873-2008.pdf
10873
10911
2008-06-06
Quantitative performance metrics for stratospheric-resolving chemistry-climate models
D. W. Waugh
V. Eyring
Department of Earth and Planetary Science, Johns Hopkins University, Baltimore, MD, USA
Deutsches Zentrum für Luft- und Raumfahrt, Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany
A set of performance metrics is applied to stratospheric-resolving
chemistry-climate models (CCMs) to quantify their ability to
reproduce key processes relevant for stratospheric ozone. The same
metrics are used to assign a quantitative measure of performance
("grade") to each model-observations comparison shown in
Eyring et al. (2006). A wide range of grades is obtained, both for
different diagnostics applied to a single model and for the same
diagnostic applied to different models, highlighting the wide range
in ability of the CCMs to simulate key processes in the
stratosphere. No model scores high or low on all tests, but
differences in the performance of models can be seen, especially for
transport processes where several models get low grades on multiple
tests. The grades are used to assign relative weights to the CCM
projections of 21st century total ozone.
However, only small differences are found between weighted
and unweighted multi-model mean total ozone projections.
This study raises several issues with the grading and weighting of
CCMs that need further examination, but it does provide a framework
that will enable quantification of model improvements and assignment
of relative weights to the model projections.
Akiyoshi, H., Sugita, T., Kanzawa, H., and Kawamoto, N.: Ozone perturbations in the Arctic summer lower stratosphere as a reflection of NO<sub>x</sub> chemistry and planetary scale wave activity, J. Geophys. Res., 109, D03304, doi:10.1029/2003JD003632, 2004.
Austin, J.: A three-dimensional coupled chemistry-climate model simulation of past stratospheric trends, J. Atmos. Sci., 59, 218–232, 2002.
Austin, J., Shindell, D., Beagley, S. R., Bruhl, C., Dameris, M., Manzini, E., Nagashima, T., Newman, P., Pawson, S., Pitari, G., Rozanov, E., Schnadt, C., and Shepherd, T. G.: Uncertainties and assessments of chemistry-climate models of the stratosphere, Atmos. Chem. Phys., 3, 1–27, 2003.
Austin, J., Wilson, R. J., Li, F., and Vomel, H.: Evolution of water vapor concentrations and stratospheric age of air in coupled chemistry-climate model simulations, J. Atmos. Sci., 64, 905–921, 2006.
Brunner, D., Staehlin, J., Rogers, H. L., et al.: An evaluation of the performace of chemistry transport models by comparison with research aircraft observations – Part 1: Concepts and overall model performance, Atmos. Chem. Phys., 3, 1609–1631, 2003
Connolley, W. M. and Bracegirdle, T. J.: An Antarctic assessment of IPCC AR4 climate models, Geophys. Res. Lett., 34, L22505, doi:10.1029/2007GL031648, 2007.
Dameris, M., Grewe, V., Ponater, M., Deckert, R., Eyring, V., Mager, F., Matthes, S., Schnadt, C., Stenke, A., Steil, B., Brühl, C., and Giorgetta, M.: Long-term changes and variability in a transient simulation with a chemistry-climate model employing realistic forcings, Atmos. Chem. Phys., 5, 2121–2145, 2005. %
de Grandpré, J., % S. R. Beagley, V. I. Fomichev, E. Griffioen, J. C. McConnell, % A. S. Medvedev, and T. G. Shepherd: Ozone climatology using % interactive chemistry: Results from the Canadian Middle Atmosphere % Model, J. Geophys. Res., 105, 26,475-26,492, 2000.
Douglass, A. R., Prather, M. J., Hall, T. M., Strahan, S. E., Rasch, P. J., Sparling, L. C., Coy, L., and Rodriguez, J. M.: Choosing meteorological input for the global modeling initiative assessment of high-speed aircraft, J. Geophys. Res., 104, 27 545–27 564, 1999.
Egorova, T., Rozanov, E., Zubov, V., Manzini, E., Schmutz, W., and Peter, T.: Chemistry-climate model SOCOL: a validation of the present-day climatology, Atmos. Chem. Phys., 5, 1557–1576, 2005.
Eyring, V., Harris, N. R. P., Rex, M., et al.: A strategy for process-oriented validation of coupled chemistry-climate models, Bull. Am. Meteorol. Soc., 86, 1117–1133, 2005.
Eyring, V., Butchart, N., Waugh, D. W., et al.: Assessment of temperature, trace species, and ozone in chemistry-climate model simulations of the recent past, J. Geophys. Res., 111, D22308, doi:10.1029/2006JD007327, 2006.
Eyring, V., Waugh, D. W., Bodeker, G. E., et al.: Multimodel projections of stratospheric ozone in the 21st century, J. Geophys. Res., 112, D16303, doi:10.1029/2006JD008332, 2007.
Fomichev, V. I., Jonsson, A. I., de Grandpré, J., Beagley, S. R., McLandress, C., Semeniuk, K., and Shepherd, T. G.: Response of the middle atmosphere to CO<sub>2</sub> doubling: Results from the Canadian Middle Atmosphere Model, J. Climate, 20, 1121–1144, 2007.
Garcia, R. R., Marsh, D., Kinnison, D., Boville, B., and Sassi, F.: Simulations of secular trends in the middle atmosphere, 1950-2003, J. Geophys. Res., 112, D09301, doi:10.1029/2006JD007485, 2007.
Gelman, M. E., Miller, A. J., Johnson, K. W., and Nagatani, R.: Detection of long-term trends in global stratospheric temperature from NMC analyses derived from NOAA satellite data, Adv. Space Res., 6, 17–26, 1996.
Gettelman, A., Birner, T., Eyring, V., Akiyoshi, H., Plummer, D. A., Dameris, M., Bekki, S., Lefevre, F., Lott, F., Brühl, C., Shibata, K., Rozanov, E., Mancini, E., Pitari, G., Struthers, H., Tian, W., and Kinnison, D. E.: The Tropical Tropopause Layer 1960-2100, Atmos. Chem. Phys. Discuss., 8, 1367–1413, 2008.
Gleckler, P.J, Taylor, K. E., and Doutriaux, C.: Performance Metrics for Climate Models, J. Geophys. Res., 113, D06104, doi: 10.1029/2007JD008972, 2008.
Grooß, J.-U. and Russell III, J. M.: Technical note: A stratospheric climatology for O<sub>3</sub>, H<sub>2</sub>O, CH<sub>4</sub>, NO<sub>x</sub>, HCl and HF derived from HALOE measurements, Atmos. Chem. Phys., 5, 2797–2807, 2005.
Hall, T. M., Waugh, D. W., Boering, K. A., and Plumb, R. A.: Evaluation of transport in stratospheric models, J. Geophys. Res., 104, 18 815–18 840, 1999.
Lott, F., Fairhead, L., Hourdin, F., and Levan P.: The stratospheric version of LMDz: Dynamical Climatologies, Arctic Oscillation, and Impact on the Surface Climate, Climate Dynamics, 25, 851–868, doi:10.1007/s00382-005-0064, 2005.
Mote, P. W., Rosenlof, K. H., McIntyre, M. E., Carr, E. S., Gille, J. C., Holton, J. R., Kinnersley, J. S., Pumphrey, H. C., Russell III, J. M., and Waters, J. W.: An atmospheric tape recorder: The imprint of tropical tropopause temperatures on stratospheric water vapor, J. Geophys. Res., 101, 3989–4006, doi:10.1029/95JD03422, 1996.
Newman, P. A., Nash, E. R., and Rosenfield J. E.: What controls the temperature of the Arctic stratosphere during spring?, J. Geophys. Res., 106, 19 999–20 010, 2001.
Pawson, S., Stolarski, R. S., Douglass, A. R., Newman, P. A., Nielsen, J. E., Frith, S. M., and Gupta, M. L.: Goddard Earth Observing System Chemistry-Climate Model Simulations of Stratospheric Ozone-Temperature Coupling Between 1950 and 2005, J. Geophys. Res, in press, 2008.
Pitari, G., Mancini, E., Rizi, V., and Shindell, D.: Feedback of future climate and sulfur emission changes an stratospheric aerosols and ozone, J. Atmos. Sci., 59, 414–440, 2002.
Randel, W., Udelhofen, P., Fleming, E., et al.: The SPARC Intercomparison of Middle-Atmosphere Climatologies. J. Climate, 17, 986–1003, 2004.
Reichler, T. and Kim, J.: How well do coupled models simulate today's climate?, Bull. Am. Meteor. Soc., 89, 303–311, doi:10.1175/BAMS-89-3-303, 2008. %
Santee, M. L., % L. Frodivaux, G. Manney, W. Read, J. Waters, M. Chipperfield, A. % Roche, J. Kummer, J. Mergentaller, and J. Russell III: Chlorine % activation in the lower stratospheric polar region during late % winter: Results from UARS, J. Geophys. Res., 101, 18835-18859, % 1996.
Shibata, K. and Deushi, M.: Partitioning between resolved wave forcing and unresolved gravity wave forcing to the quasi-biennial oscillation as revealed with a coupled chemistry-climate model, Geophys. Res. Lett., 32, L12820, doi:10.1029/2005GL022885, 2005.
Schmittner, A., Latif, M., and Schneider B.: Model projections of the North Atlantic thermohaline circulation for the 21st century, Geophys. Res. Lett., 32, L23710, doi:10.1029/2005GL024368, 2005.
Steil, B., Brühl, C., Manzini, E., Crutzen, P. J., Lelieveld, J., Rasch, P. J., Roeckner, E., and Krüger, K.: A new interactive chemistry climate model. 1: Present day climatology and interannual variability of the middle atmosphere using the model and 9 years of HALOE/UARS data, J. Geophys. Res., 108, 4290, doi:10.1029/2002JD002971, 2003.
Stevenson, D. S., Dentener, F. J., Schultz, M. G., et al.: Multimodel ensemble simulations of present-day and near-future tropospheric ozone, J. Geophys. Res., 111, D08301, doi:10.1029/2005JD006338, 2006.
Strahan, S. E. and Douglass, A. R.: Evaluating the credibility of transport processes in simulations of ozone recovery using the Global Modeling Initiative three-dimensional model, J. Geophys. Res., 109, D05110, doi:10.1029/2003JD004238, 2004.
Swinbank, R. and O' Neill, A.: A stratosphere-troposphere data assimilation system, Monthly Weather Review, 122, 686–702, 1994.
Taylor, K. E.: Summarizing multiple aspects of model performance in a single diagram, J. Geophys. Res., 106, 7183–7192, 2001.
Tian, W. and Chipperfield, M. P.: A new coupled chemistry-climate model for the stratosphere: The importance of coupling for future O<sub>3</sub>-climate predictions, Quart. J. Roy. Meteor. Soc., 131, 281–304, 2005.
Uppala, S., Kållberg, P. W., Simmons, A. J., et al.: The ERA-40 reanalysis, Quart. J. Roy. Meteor. Soc., 131, 2961–3012, 2005.
Waugh, D. W. and Hall, T. M.: Age of stratospheric air: theory, observations, and models, Rev. Geophys., 40, 1010, doi:10.1029/2000RG000101, 2002.
Wilks, D. S.: Statistical methods in the atmospheric sciences, Academic Press, Lond, 467 pp., 1995. %
% World Meteorological Organization (WMO)/United Nations Environment % Programme (UNEP), Scientific Assessment of Ozone Depletion: % 2002, World Meteorological Organization, Global Ozone Research and % Monitoring Project, Report No. 47, Geneva, Switzerland, 2003.
World Meteorological Organization (WMO)/United Nations Environment Programme (UNEP): Scientific Assessment of Ozone Depletion: 2006, World Meteorological Organization, Global Ozone Research and Monitoring Project, Report No 50, Geneva, Switzerland, 2007.