This presentation describes the purpose of performance metrics as they relate to the analysis of climate models.
To demonstrate the following:
- The usefulness of any particular set of metrics depends on the application.
- The relationship between a climate model’s skill in simulating present conditions and its predictive reliability is largely unknown.
- At this time, there is little justification for reliance on any single metric to gauge simulation quality
ACTIVITY DESCRIPTION AND TEACHING MATERIALS
This presentation addresses the following points:
- Background, orientation, definitions
- Uses of metrics
- What’s nexts next?
View Full Presentation >> "Uses of Metrics in the Evaluation and Application of Climate Models"
TEACHING NOTES / CONTEXT FOR USE
Challenges faced in evaluating climate models: Variety of fields to consider
•Multitude of phenomena / processes of interest
•Range of time and space scales
•A choice of error measures (e.g., RMS error, correlation)
•Limited opportunities for verification of “forecasts”
•Must account for observational errors/uncertainty
•Some aspects of simulations are not deterministic
Fundamental research questions:
What is the relationship between skill in simulating observed phenomenon and (unobserved) future climate?
- “Perfect model” experiments
- Identification of processes critical to future climate change that can be thoroughly validated on shorter time-scales
- For a given application, is there some minimum set of metrics that can be objectively justified for gauging climate model reliability?
- Can we justifiably construct a single metric
- To gauge reliability of individual model predictions?
- To produce an optimally-weighted consensus prediction?