The influence of single model ensemble on the simulated extratropical interannual variability

This study compares the interannual variance of boreal winter near-surface temperature (DJF T2m) with and without performing single model ensemble (SME) in seasonal hindcasts (DEMETER, ENSEMBLES, and NCEP CFSv2) and historical climate simulations (CMIP5). The results demonstrate that the extratropical temperature variability is significantly reduced after performing SME even though the signal in the tropical Pacific remains strong. Cancellation between positive and negative perturbations simulated by individual model members, of both tropical and extratropical origins, leads to the under-simulation. The atmospheric circulation induced by tropical Pacific sea surface temperature is not well represented in global climate models and the simulation is further deteriorated by SME, leading to an unrealistically weak interannual variance of simulated winter temperature in North America. Similar effect was also found in North Eurasia where winter temperature is strongly influenced by atmospheric internal variability and its interaction with land and ice/snow in the middle-high latitudes. The SME procedure should be avoided when evaluating the model performance in simulating the higher-order long-term statistics (such as variance). Variance of individual models should be calculated first and then averaged among members. Models used in seasonal forecast and long-term climate simulation already have good capability in simulating the long-term statistics of stochastic processes in the extratropics, although the capability in accurately simulating the temporal variation is still poor. Key word: single model ensemble; interannual variability; climate simulation


Introduction
The interannual variability is one of the climate variations that strongly influence human activities and can be simulated much better by general circulation models (GCMs) than other climate variations.Therefore, interannual variability has been extensively studied for both seasonal forecasts and climate simulations purpose.The seasonal forecasts have been shown to have good and stable predictability, especially for the winter season during which the most important influencing factor is the El Niño and Southern Oscillation (ENSO) (e.g., Weisheimer et al. 2009).However, extreme climatic events of temperature on seasonal timescale, especially in the extratropics, have been observed to increase in recent years (Palmer 2014;Lee et al. 2015).The ability in simulating and forecasting the extratropical interannual variability that may arise due to factors other than ENSO is relatively unexplored.
Several studies have suggested that the cold winter extremes in the northern continents may be associated with Arctic sea-ice loss (e.g., Cohen et al. 2014;Mori et al. 2014;Lee et al. 2015; and reference therein), but large uncertainties remain due to our limited knowledge about the climate system.It is therefore important to understand the ability of the current state-of-the-art GCMs in simulating and forecasting the interannual variability in the extratropics.In this study, we evaluate the performance of climate models in this aspect using boreal winter (Dec, Jan, Feb) near-surface (2-meter height) temperature (DJF T2m).The result shows that SME significantly decreases the interannual variability in both seasonal forecasts and climate simulations, especially in the extratropics.This work has been accepted by Terrestrial, Atmospheric and Oceanic Sciences.Please find the completed research report in an upcoming issue.

Data and Methods
We use datasets from four different projects.The first two were both provided by European Center, which are Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER) project (Palmer et al. 2004) and EMSEMBLES (Weisheimer et al. 2009).See Table 1 and Figure 1 in Palmer et al. 2004 for the details of the DEMETER project and Table 1 in Weisheimer et al. 2009 for the details of the EMSEMBLES project.For both datasets, the DJF forecasts initialized at Nov. 1 st are used in this study.A more recent dataset, the National Centers for Environmental Prediction (NCEP) climate forecast system version 2 (CFSv2), is used to provide an updated vision of the seasonal forecast (Saha et al. 2014).We use the monthly mean time series initialized at Oct 29 th and Nov. 2 nd that provides 8 ensemble members for DJF forecasts.For the long-term climate simulations, the World Climate Research Programme's (WCRP's) Coupled model intercomparison project phase 5 (CMIP5) (Taylor et al. 2012) historical simulations are carried out from 1850 to 2005 forced by historical emissions.Models that provide only one ensemble member are excluded and therefore there are only 20 models in this study.For comparison, the ERA40 DJF T2m is used to evaluate the performance of model simulations.
We first calculate the single model ensemble of DJF seasonal mean for each model.Because we are focusing on interannual timescale, the 9-year running mean is subtracted to remove variations with periods longer than this timescale.The temporal variance of the remaining time series is then calculated to obtain the interannual variability.The multi-model mean variance of each project is calculated at the end, except CFSv2 because the system consists of only one model.We use the mean variance of multi models to indicate the overall performance of the interannual variability in each project.The ensemble average calculated through these steps is indicated as SME+MME.In order to show the impact of SME on interannual variability, we calculate ensemble average without SME, meaning all ensemble members are treated as individuals, and the ensemble mean of all individual members is indicated as MME.Note that both SME and MME can decrease the amplitude of interannual variability if the ensemble step is placed before the computation of variance.The reason that we emphasize on SME is because it is often placed at the first step in data processing.

The Interannual Variability of DJF T2m
The spatial pattern of DJF T2m interannual variances is shown in Figure 1.In ERA-40 (Figure 1a), the variance is large over the NH continents and tropical eastern Pacific, owing to the influences from extratropical large-scale circulation systems and the ENSO, respectively.
After the standard ensemble procedure (SME+MME), the simulated variance over the NH continents is significantly weaker than observed in all hindcast datasets (Figure 1b-e).The variance of ENSEMBLES (Figure 1c) and CFSv2 (Figure 1d) is particularly weak.The tropical variance associated with ENSO is overestimated in DEMETER, but it is close to ERA40 in ENSEMBLES and CFSv2.The interannual variance of ENSO is well captured because the ENSO simulation is highly constrained by the slow evolving SST assimilated into seasonal forecasts.By contrast, in the CMIP5 ensemble (Figure 1e), the interannual variance in the tropical eastern Pacific nearly disappears after the SME procedure.It can be easily understood that the occurring year and amplitude of ENSO events in the long-term climate simulations do not necessarily synchronize between members due to the internal variability of the coupled atmosphere-ocean system and the signals will be partially canceled during the SME procedure.
If the ensemble is made without the SME procedure (i.e., MME, Figure 1 right panel), the extratropical variance in all projects and the tropical variance in CMIP5 are well retained.The difference between the two methods reveals that similar cancelation effect as for the tropical Pacific signal in CMIP5 may be involved for extratropical T2m during the SME procedure because the extratropical temperature fluctuation is less regulated by the tropical oceans.The impact of SME on the variance can be more than half of the amplitude.By contrast, the variance in the tropical Pacific shows very small difference with and without the SME procedure in seasonal forecasts, again reflecting the continuous influence of initial condition.
In order to show the influence from SME on each member, Figure 2 compares the area-averaged variances of each simulation with ERA40.The box and whisker indicate the median and spread of variance of all model members (see figure caption for the details).For the extratropical region (Figure 2a), most of the models simulated variances comparable to the ERA40 (the horizontal dash line), except the much larger value from CFSv2.However, the variance of T2m with SME (red open circles) drops significantly in all projects.As for tropical Pacific variance (Figure 2b), models in both ENSEMBLE and CFSv2 produced reasonable simulation, while those in DEMETER tended to simulate much larger variance comparing to ERA40.In contrast to the extratropical variance, the SME procedure does not affect the performance in the tropical Pacific (red open circles in Figure 2b) because of the strong influence of initial condition.The contrasted performance in the tropics and extratropics suggests that good ENSO simulation (forecast) does not necessarily yield reliable simulation in the extratropics.The median of tropical variance simulated by CMIP5 models is close to the observed but with a large spread.The influence of SME for CMIP5 tropical Pacific is seen clearly again as the values of SME (red open circles) are much smaller than the variance of individual members (the box and whisker) and ERA40 (the horizontal dash line).The results presented in this section indicate that if the interannual variance of individual member is calculated individually (i.e., the MME calculation in this study), the extratropical variance can be retained much better in seasonal forecast.For climate simulation (CMIP5 project), both extratropical and tropical variances are kept better as shown in Figure 1.
The connection between regional temperature and other physical variables can be easily shown by the cross-covariance maps (Figure 3). Figure 3a shows the covariance between T2m averaged over North America (40°N-70°N, 120°W-40°W) and T2m at all other grid points.The figure shows that North American T2m closely varies with tropical Pacific T2m, meaning the signal of thermal forcing is from the tropics.Figure 3b shows that North American T2m is associated with North Pacific Z500, which is part of the wave-like pattern originating from the tropical Pacific, but not associated with Arctic Z500.The same calculation is carried out for the North Eurasian continent (40°N-70°N, 1°E-150°E) as well (Figure 3c and d).In contrast to North America, Figure 3c shows very low covariance between the North Eurasia and the tropical Pacific, indicating minimum influences from ENSO events.However, the North Eurasian T2m is highly associated with the Arctic pressure variation and anomaly over the North Atlantic (Figure 3d), suggesting the influences from high-latitude systems.A similar result was shown in Fig. 3a in Higgins et al. (2002).These high-latitude influences have been suggested to be associated with Arctic sea ice loss and Arctic Oscillation or North Atlantic Oscillation (AO/NAO).
The performance of each project in turns of simulating the physical processes associated with extratropical T2m is also evaluated by calculating the covariance maps (the same as Figure 3, but for models, figures are not shown).Our results show that SME smooths out the amplitudes of atmospheric perturbations in all seasonal forecast projects.All of the processes mentioned previously are not simulated well enough in GCMs.However, the covariances in CMIP5 project are significantly larger than those in the seasonal forecast projects.This implies that the signals of atmospheric internal modes may be retained better in long-term climate simulations than in seasonal forecast, but the exact reasons need further investigation.Still, the damping effect from SME is also seen in simulations.

Conclusions
The standard ensemble procedure is usually done with SME first and then MME.The evaluation of seasonal forecast considers mostly the mean state either from single model or multi models.The SME procedure may not influence the evaluation of the mean state significantly but have strong impact on the variance (or interannual variability).The reason is likely due to the cancellation between positive and negative anomalies between ensemble members during the SME procedure.Therefore, a seasonal forecast can be evaluated as good in terms of the mean, but not in the interannual variance.Such cancellation effect is more significant in the extratropics where atmospheric internal variability is larger than in the tropics where the ENSO dominates.This is much less a problem in the tropics for the seasonal forecasts that are initiated with well-observed initial condition in both atmosphere and ocean.As it has been known that the well-simulated ENSO and tropical temperature contribute to most of the forecast skill (Weisheimer et al., 2009;Alessandri et al., 2011) in the tropical variability.By contrast, the cancellation effect in the tropics remains significant for CMIP type simulations because of independence of initial conditions.
The extratropical temperature variances are contributed from both tropical and extratropical phenomena, i.e., the wave-like perturbations originating from the tropics and the perturbations of extratropical origin such as AO, NAO, and the Arctic sea ice.The SME procedure will smear out the amplitude of forecasted perturbation, even when the tropical SST signals were strong.The atmospheric circulation induced by tropical Pacific SST is not well represented in global climate models and the simulation is further deteriorated by SME, leading to an unrealistically weak interannual variance of simulated winter temperature in North America.Similar effect was also found in North Eurasia where winter temperature is strongly influenced by atmospheric internal variability and its interaction with land and ice/snow in the middle-high latitudes.
The results of this study suggest that the SME procedure should be avoided when evaluating the model performance in simulating the higher-order long-term statistics (such as variance).Variance of individual models should be calculated first and then averaged among members.Another challenge is the Arctic amplification that has been suggested to be associated with the global warming tendency and may have become a more important factor that influences the extratropical temperature and climate in general.Skillful forecast of ENSO, which has been the major focus during the development of seasonal forecast models in the past few decades, is no longer sufficient for further improvement.Better understanding of the extratropical variability such as the AO/NAO and the influence of Arctic sea ice and the ability to simulate their influences will definitely help improve the extratropical seasonal forecast and long-term climate simulation.

Figure 1 .
Figure 1.The interannual variability of boreal winter near-surface temperature (DJF T2m).The top row shows ERA40.Results from each project are shown below with the project name indicated on the upper-left corners.The left panel shows ensemble variances using SME+MME method and the right panel shows results using MME.

Figure 2 .
Figure 2. The spread of area-averaged variances of all simulations (indicated by the box-and-whisker representation), comparing to SME (red open circles) and ERA40 (the dash line), for a) the NH extratropics (30°N-90°N, 0°-360°) and b) the tropical Pacific (15°S-15°N, 150°E-80°W).The whisker lower and upper ends indicate the minimum and maximum of the ensemble spread, the line in the box marks the ensemble median, and the box bottom and top are the first and third quartiles.

Figure 3 .
Figure 3.The cross-covariance between T2m (averaged over selected regions) and selected physical variables at each grid point, using the seasonal mean time series of ERA40 from 1980 to 2000.(a) Covariance between North American T2m (40°N-70°N, 120°W-40°W) and T2m at each grid point; (b) North American T2m and Z500 at each grid point; (c) North Eurasian T2m (40°N-70°N, 1°E -150°E) and T2m at each grid point; (d) North Eurasian T2m and Z500 at each grid point.Note the color ranges are different in the left and right columns.