Mesoscale Model Evaluation课件.ppt

Mesoscale Model Evaluation Mike Baldwin Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma Also affiliated with NOAA/NSSL and NOAA/NWS/SPC NWS – forecasts on hi-res grids What would you suggest that NWS do to verify these forecasts? Issues in mesoscale verification Validate natural behavior of forecasts Realistic variability, structure of fields Do predicted events occur with realistic frequency? Do characteristics of phenomena mimic those found in nature? Traditional objective verification techniques are not able to address these issues Outline Problems with traditional verification Solutions: Verify characteristics of phenomena Verify structure/variability Design verification systems that address value of forecasts Traditional verification Compare a collection of matching pairs of forecast and observed values at the same set of points in space/time Compute various measures of accuracy: RMSE, bias, equitable threat score A couple of numbers may represent the accuracy of millions of model grid points, thousands of cases, hundreds of meteorological events Boiling down that much information into one or two numbers is not very meaningful Dimensionality of verification info Murphy (1991) and others highlight danger of simplifying complex verification information High-dimension information = data overload Verification information should be easy to understand Need to find ways to measure specific aspects of performance Quality vs. value Scores typically measure quality, or degree in which forecasts and observations agree Forecast value is benefit of forecast information to decision maker Value is subjective, complex function of quality High-quality forecast may be of low value and vice versa Forecast #1: smooth Traditional “measures-oriented” approach to verifying these forecasts Phase/timing errors High-amplitude, small-scale forecast and observed fields are most sensitive to timing/phase errors Mean Squared Error (MSE) For 1 point phase