Atmospheric Chemistry and Physics Discussions www.atmos-chem-phys-discuss.net 1680-7367 1680-7375 9 3 2009 10.5194/acpd-9-14141-2009 http://www.atmos-chem-phys-discuss.net/9/14141/2009/ http://www.atmos-chem-phys-discuss.net/9/14141/2009/acpd-9-14141-2009.html http://www.atmos-chem-phys-discuss.net/9/14141/2009/acpd-9-14141-2009.pdf 14141 14164 2009-06-26 Comment on "Quantitative performance metrics for stratospheric-resolving chemistry-climate models" by Waugh and Eyring V. Grewe volker.grewe@dlr.de R. Sausen Deutsches Zentrum für Luft- und Raumfahrt, Institut für Physik der Atmosphäre, Oberpfaffenhofen, 82230 Wessling, Germany This comment focuses on the statistical limitations of a model grading, as applied by D. Waugh and V. Eyring (2008) (WE08). The grade <i>g</i> is calculated for a specific diagnostic, which basically relates the difference of model and observational data to the standard deviation in the observational dataset. Monte Carlo simulations show that this method is not leading to statistical significant gradings. Moreover, the difference between two models is hardly significant. The results of the statistical tests performed in WE08 agree with our findings. However, most of those tests are based on special cases, which implicitely assume that observations are available without any errors and that the interannual variability of the observational data and the model data are equal. Without these assumptions the grading becomes basically insignificant. We further show that the inclusion of confidence intervals into the grading approach is necessary, since it has the potential to change the grading results drastically. Lary, D. J. and Aulov, O.: Space-based measurements of HCl: Intercomparison and historical contex, J Geophys Res., 113, D15S04, doi:10.1029/2007JD008715, 2008. Waugh, D. W. and Eyring, V.: Quantitative performance metrics for stratospheric-resolving chemistry-climate models, Atmos. Chem. Phys., 8, 5699–5713, 2008. World Meteorological Organisation (WMO): Scientfic Assessment of Ozone Depletion, Geneva, 2006.