Report on the Third SPARC CCMVal Workshop

 26-28 June 2007, University of Leeds, United Kingdom

V. Eyring, DLR, Germany (veronika.eyring@dlr.de)

A. Gettelman, NCAR, USA (andrew@ucar.edu)

N.R.P. Harris, University of Cambridge, UK (Neil.Harris@ozone-sec.ch.cam.ac.uk)

S. Pawson, NASA Goddard, USA (Steven.Pawson-1@nasa.gov)

T.G. Shepherd, University of Toronto, Canada (tgs@atmosp.physics.utoronto.ca)

D.W. Waugh, John Hopkins University, USA (waugh@jhu.edu)

H. Akiyoshi, NIES, Japan (hakiyosi@nies.go.jp)

N. Butchart, Met Office, UK (neal.butchart@metoffice.gov.uk)

M.P. Chipperfield, University of Leeds, UK (martyn@env.leeds.ac.uk)

M. Dameris DLR, Germany (Martin.Dameris@dlr.de)

D.W. Fahey, NOAA-ESRL, USA (David.W.Fahey@noaa.gov)

P.M. de F. Forster, University of Leeds, UK (piers@env.leeds.ac.uk)

P.A. Newman, NASA Goddard, USA (Paul.A.Newman@nasa.gov)

M. Rex, AWI-Potsdam, Germany (mrex@awi-potsdam.de)

R.J. Salawitch, NASA JPL, USA (rjs@caesar.jpl.nasa.gov)

B.D. Santer, LLNL, USA (santer1@llnl.gov).

Introduction and Rationale

The coupling of stratospheric chemical models with climate models has led to a new generation of models far more complex than those available when the Montreal Protocol was signed twenty years ago.  This increased complexity allows questions about future stratospheric ozone and UV radiation levels to be studied in much more detail than could be done at that time.  However, the workings of these chemistry-climate models (CCMs) themselves are also much harder to fully understand.  Periodic assessments of the family of stratospheric CCMs and their general circulation models (GCMs) have been organised under the auspices of the SPARC GRIPS and CCMVal activities (Pawson et al., 2000; Eyring et al., 2005) and have contributed directly to the evaluation of CCMs during the preparation of the UNEP/WMO Scientific Assessments of Ozone Depletion (Eyring et al., 2006, 2007).  However, there is insufficient time to evaluate CCM performance thoroughly while preparing the Ozone Assessments.  For this reason SPARC CCMVal is undertaking to prepare a Report on the Evaluation of Chemistry Climate Models by 2009 in time for consideration in the anticipated UNEP/WMO Ozone Assessment in 2010.  The SPARC CCMVal report itself has two major aims: 1) provide valuable base material for that Assessment, and 2) improve the understanding of the strengths and weaknesses of CCMs and thus increase their integrity and credibility.

The third CCMVal workshop was held at the University of Leeds in June 2007, following on from earlier workshops in Grainau, Germany in 2003 and Boulder, USA in 2005.  The aims of the workshop were: (a) to discuss recent advances in model development and the means to diagnose them; and (b) to prepare for the SPARC Report on the Evaluation of Chemistry Climate Models. Approximately eighty members of the atmospheric and climate communities from Europe, North America, Japan and New Zealand attended the workshop.  The attendees included representatives from nearly all the major stratospheric CCM groups in the world. The agenda, abstracts and a list of participants can be found at the workshop’s website (http://www.see.leeds.ac.uk/ccmval2007).

Science sessions

A total of 25 oral and 41 poster presentations were made which focussed on research related to CCMVal activities.  The presentations included studies using existing diagnostics for assessing particular processes in CCMs and proposals for new diagnostics.  Many were based on the data that is already present in the CCMVal data archive at the British Atmospheric Data Centre (BADC).  Presentations focused on four main areas:

In addition to presentations on model analysis and validation, results of recent CCM simulations were presented and new applications of CCMs were discussed.  Programmatic presentations on other international programmes, such as SPARC DynVar (see SPARC Newsletter No. 29) and SOLARIS, were given to clarify their relation to CCMVal.

Top

SPARC CCMVal Report on the Evaluation of Chemistry Climate Models

The SPARC CCMVal report will consist of two major parts. Part A will evaluate how well the CCMs perform according to the CCMVal diagnostics tables under present-day conditions in five major areas (radiation, dynamics, transport, stratospheric chemistry and microphysics, and upper troposphere / lower troposphere).  Each process is associated with one or more model diagnostics and with relevant data sets that can be used for model evaluation.  Due to a lack of appropriate measurements, the evaluation of the radiation descriptions will be largely based on detailed comparisons of the radiation codes from the participating models.  The chapters in Part A will also include long-term changes of the key processes in the past and future (e.g changes in the Brewer-Dobson circulation, PSC frequency, sudden warmings).  This approach provides a coherent framework for the evaluation of CCMs and will be used as a basis for the assessment in Part B.

Part B will examine the coupled ozone-climate response to natural and anthropogenic forcing.  The chapter on natural variability will evaluate how well CCMs represent the effects of various sources of coherent forced and unforced natural variability (QBO, volcanic, solar, ENSO) on stratospheric dynamics, radiation, chemistry and transport.  The chapter on long-term projections of stratospheric ozone will focus on simulated long-term changes in ozone and the causes of these changes (i.e. relate to changes in chemistry, dynamics, radiation, transport and UTLS discussed in Part A). The chapter on the effect of the stratosphere on the troposphere will include the radiative forcing from ozone changes, tropospheric effects of polar ozone depletion, and changes in the flux of ozone to the troposphere over long timescales.

New CCMVal Reference Scenarios

On the first day of the workshop, a breakout group met to discuss new model simulations that would be tailored to the SPARC Report on Evaluation of Chemistry Climate Models.  The group recommended three different reference simulations that could be run by the various modelling groups.  The first simulation (REF0) is defined as a time-slice experiment characteristic of 2000 levels. This run will provide a basic assessment of chemical and dynamical conditions in the models during a period of peak ozone loss. It should be possible for analysis of runs based on REF0 to start much earlier than the other scenarios, which will be useful for developing the diagnostics as well as providing a preliminary evaluation of the models.  The second simulation (REF1) is defined as a transient run from 1960 to the present. This scenario is meant to simulate the past climate and would be evaluated by comparing to observations. REF1 includes observed sea surface temperatures, volcanic aerosols, and solar forcings.  The third simulation (REF2) is a transient run from 1960 to 2100. This scenario will simulate both the past and future in a consistent manner, but with a primary focus on ODS and GHG forcing. REF2 will include fixed background aerosol conditions that excluded volcanic forcings, and sea surface temperatures from a coupled atmosphere-ocean model simulation or from the CCM itself if coupled to an ocean model.  In addition several sensitivity experiments have been proposed. An overview of the proposed CCMVal reference and sensitivity simulations is given in Eyring et al. (2008). The specified forcings for the new reference simulations and a detailed description will be made available for download on the CCMVal website.

Data and Tools

A working group on data and tools discussed several issues regarding using model output, future model output, diagnostics and observations.  All agreed that the CCMVal archive at BADC is working well for CCMVal modellers and collaborators.  There are other options to explore improved (e.g. web based) interfaces to the data and improved data transfer (such as sub-setting).  It was agreed to move towards a Climate and Forecast (CF) standard compliant NetCDF format for future data requests. The group discussed output for future diagnostics and agreed that the base output for core diagnostics should be three-dimensional (latitude, longitude, pressure) monthly mean fields.  The vertical coordinate should be pressure based, but there was no consensus whether the pressure levels should be standard for all models or model specific.  Several derived fields are desirable, e.g. Eliassen-Palm (EP) fluxes, potential vorticity (PV) and tropopause characteristics.  Some high frequency diagnostics (such as instantaneous snapshots of chemical fields) will probably be requested by some chapters.  Details on which fields at what frequency are likely to be determined by the requirements of the SPARC CCMVal report and will be finalised in 2007.

The diagnostic tools and observations used for model evaluation were discussed.  It was agreed that a common diagnostic tool for evaluating climatological fields (such as those appearing in Eyring et al., 2006, 2007) would be valuable for quick assessment and comparison of basic model performance prior to more detailed analysis.  Such a tool should be based on open source formats and might be designed to run on a server linked to the CCMVal archive.  Additional resources would be required to build such a tool and are requested in some proposals currently pending in the USA and Europe.  There was general agreement to prepare observational data sets in the same format as the model output and to add these to the model output archive. These would include individual instruments as well as composite data sets.

Evaluation

CCMVal was formulated to evaluate the skill of CCMs being used for projections of stratospheric ozone.  A comprehensive list of diagnostics of model performance has been developed previously for radiation, dynamics, transport, and chemistry.  The workshop participants discussed how to approach the next step; namely the evaluation of individual models for their performance in each diagnostic category. 

During a breakout group discussion, the following recommendations were put forward:

1.      Establish guidelines for models participating in any chemistry-climate experiments that relate to basic model features, e.g., stratospheric chemistry, polar processes.

2.      Develop a quantitative metric of performance (grade) for each model for each diagnostic.  The grades are an effort to describe the skill of each model to represent important features of the atmosphere.

3.      Assign weights to the importance of each diagnostic based on what model product is being used in a model-based assessment.  For instance, if the product is the “recovery date of stratospheric ozone,” then a diagnostic relating to the behaviour of Cly might get high weight, while the skill of predicting 500-hPa geopotential height may receive a lower weight. 

4.      Convolve the diagnostic weights and grades for each model to define an overall weight for a specific product. 

5.      Calculate a weighted-ensemble average and uncertainty for a product using the set of model weights to define best estimates and variances (spreads) for model products, such as stratospheric ozone projections.  

This is an ambitious goal and there was considerable discussion about how well it can be attained.  However, there was also general agreement that the goal is a worthy one and that even trying to achieve it will be very interesting and worthwhile.

Link to the anticipated UNEP/WMO 2010 Assessment

The outcome of the extensive evaluation in the SPARC CCMVal report should provide the authors of the next UNEP/WMO Ozone Assessment with a sound basis to make objective judgements of the uncertainties associated with future ozone projections from the participating CCMs.  The two-way communication linking the CCM groups with CCMVal and the WMO/UNEP Ozone Assessment is illustrated in Figure 1.  CCMVal acts as a resource for the modelling groups and for the Ozone Assessment by developing and maintaining evaluation tools for the models, maintaining definitions and boundary condition data for "scenario" experiments, and archiving output data from the models.  The CCM groups will interact with CCMVal in defining and applying the evaluation tools, using the boundary condition data, and providing model output.  It is anticipated that the Ozone Assessment will make use of CCMVal resources by working with the CCM groups to help in defining relevant model scenarios, using the data bases of model outputs and applying the tools and metrics derived by CCMVal in their evaluation of model results.  In addition, the Assessment authors may solicit data from other model groups and, if they wish, may apply CCMVal diagnostic tools to evaluate these model results.  The coordination, support, and products that SPARC CCMVal provides for the CCM community represent an important additional resource for the Assessment process.

Top

Timetable

The SPARC CCMVal report needs to be finished by the end of 2009 if it is to be available for consideration in the 2010 Ozone Assessment.  The timetable for the report preparation is thus:

November 2007                      Definition of chapter outlines, scenarios and diagnostics

March 2008                             REF0 runs available for analysis        

August 2008                           Lead Author meeting coupled to SPARC General Assembly

October 2008                          All model runs completed and available for analysis

March 2009                             Draft for internal review

May 2009                                CCMVal workshop 2009 in Toronto

August –September 2009        External review

October-November 2009        Review meeting

December 2009                       Report finished

Updated information on the SPARC CCMVal Report, the new reference and sensitivity simulations and data requests can be found at the CCMVal website (http://www.pa.op.dlr.de/CCMVal/).

Figure 1.  Model of the relationships between CCMVal, the CCM groups, and the UNEP/WMO Assessment. For better resolution, please contact the SPARC IPO

Acknowledgements

The workshop was organised by the Steering Committee of the CCM Validation (CCMVal) Activity on behalf of WCRP's (World Climate Research Programme) SPARC (Stratospheric Processes and their Role in Climate). We thank Martyn Chipperfield and Piers Forster of the University of Leeds, UK for the local organization and all the workshop participants for their valuable contributions.

References

Bull. Am. Meteorol. Soc., 86, 1117–1133, 2005.

Eyring, V., et al., Assessment of temperature, trace species and ozone in chemistry-climate model simulations of the recent past, J. Geophys. Res., 111, D22308, doi:10.1029/2006JD007327, 2006.

Eyring, V., et al., Multimodel projections of stratospheric ozone in the 21st century, J. Geophys. Res., 112, D16303, doi:10.1029/2006JD008332, 2007.

Eyring, V. et al., Overview of the new CCMVal reference and sensitivity simulations in support of upcoming ozone and climate assessments and the planned SPARC CCMVal Report, SPARC Newsletter No. 30, this issue, 2008.

Pawson, S., et al., The GCM-Reality Intercomparison Project for SPARC: Scientific Issues and Initial Results, Bull. Am. Meteorol. Soc., 81, 781-796, 2000.

Top