ACPD Atmospheric Chemistry and Physics Discussions ACPD 1680-7375 Copernicus GmbH Göttingen, Germany 10.5194/acpd-9-14141-2009 Comment on "Quantitative performance metrics for stratospheric-resolving chemistry-climate models" by Waugh and Eyring Grewe V. 1 Sausen R. 1 Deutsches Zentrum für Luft- und Raumfahrt, Institut für Physik der Atmosphäre, Oberpfaffenhofen, 82230 Wessling, Germany 26 06 2009 9 3 14141 14164 This is an open-access article ditributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. This article is available from http://www.atmos-chem-phys-discuss.net/9/14141/2009/acpd-9-14141-2009.html The full text article is available as a PDF file from http://www.atmos-chem-phys-discuss.net/9/14141/2009/acpd-9-14141-2009.pdf

This comment focuses on the statistical limitations of a model grading, as applied by D. Waugh and V. Eyring (2008) (WE08). The grade <i>g</i> is calculated for a specific diagnostic, which basically relates the difference of model and observational data to the standard deviation in the observational dataset. Monte Carlo simulations show that this method is not leading to statistical significant gradings. Moreover, the difference between two models is hardly significant. The results of the statistical tests performed in WE08 agree with our findings. However, most of those tests are based on special cases, which implicitely assume that observations are available without any errors and that the interannual variability of the observational data and the model data are equal. Without these assumptions the grading becomes basically insignificant. We further show that the inclusion of confidence intervals into the grading approach is necessary, since it has the potential to change the grading results drastically.

 References Lary, D. J. and Aulov, O.: Space-based measurements of HCl: Intercomparison and historical contex, J Geophys Res., 113, D15S04, doi:10.1029/2007JD008715, 2008. Waugh, D. W. and Eyring, V.: Quantitative performance metrics for stratospheric-resolving chemistry-climate models, Atmos. Chem. Phys., 8, 5699–5713, 2008. World Meteorological Organisation (WMO): Scientfic Assessment of Ozone Depletion, Geneva, 2006.