SUMMARY

This presentation describes the purpose of performance metrics as they relate to the analysis of climate models.

GOALS

To demonstrate the following:

• The usefulness of any particular set of metrics depends on the application.
• The relationship between a climate model’s skill in simulating present conditions and its predictive reliability is largely unknown.
• At this time, there is little justification for reliance on any single metric to gauge simulation quality

ACTIVITY DESCRIPTION AND TEACHING MATERIALS

This presentation addresses the following points:

• Background, orientation, definitions
• Uses of metrics
• What’s nexts next?

View Full Presentation >> "Uses of Metrics in the Evaluation and Application of Climate Models"

TEACHING NOTES / CONTEXT FOR USE

Challenges faced in evaluating climate models: Variety of fields to consider

•Multitude of phenomena / processes of interest

•Range of time and space scales

•A choice of error measures (e.g., RMS error, correlation)

•Limited opportunities for verification of “forecasts”

•Must account for observational errors/uncertainty

•Some aspects of simulations are not deterministic

ASSESSMENT

Fundamental research questions:

What is the relationship between skill in simulating observed phenomenon and (unobserved) future climate?

• “Perfect model” experiments
• Identification of processes critical to future climate change that can be thoroughly validated on shorter time-scales
• For a given application, is there some minimum set of metrics that can be objectively justified for gauging climate model reliability?
• Can we justifiably construct a single metric
• To gauge reliability of individual model predictions?
• To produce an optimally-weighted consensus prediction?