10240-member ensemble Kalman filtering with an intermediate AGCM

We have achieved running a 3-week experiment of a 10240-member local ensemble transform Kalman filter (LETKF) with a simple low-resolution AGCM known as the SPEEDY model. The figure above shows 10240 equally-likely parallel earths, with magnified pictures. Every earth is slightly different, withing the range of uncertainties to the best of our knowledge.

The large ensemble data assimilation computations were only possible using the leading-edge K computer and the eigenvalue solver "EigenExa" that allows an extremely effective use of the K computer. We achieved amazingly high 44% efficiency, 263 TFLOPS using 4608 nodes of the K computer (about 1/20 of the full capacity). The parallel-efficient LETKF was also a must.

With 10240 members, we could obtain a very precise probabilistic representation of the earth atmosphere. Long-range error correlations beyond continental scales and bimodal structures of moisture variables were clearly represented. These were very difficult to observe with less than a few hundred members, a typical choice in ensemble data assimilation of the global atmosphere.

For more details, refer to the press release on this research achievement. Here are the links to the press release and the original research article published in Geophysical Research Letters.

• RIKEN Press Release (English, Japanese)

• Miyoshi, T., K. Kondo, and T. Imamura, 2014: The 10240-member ensemble Kalman filtering with an intermediate AGCM. Geophys. Res. Lett., 41, doi:10.1002/2014GL060863.

A multi-scale localization method was developed

We developed a new approach to error-covariance localization that considers multi-scale structure of the forecast error covariance in ensemble data assimilation methods. We found that this new approach is practical and very effective in improving accuracy of ensemble data assimilation.

With higher-resolution models, we tend to have more sampling noise in the shorter range due to limited ensemble size. This limits the use of observation data only in a limited range even though the data should impact a larger area. The figure above illustrates how the proposed multi-scale localization works. The patterns indicate what impact the observation data at the star point have. We mix the shorter-range impact (top left) with the longer-range impact (bottom left) to obtain a hybrid of both (right). This way, we preserve more structure in the shorter range, while we still include impact on an extended area using the smoothed error covariance (or smoothed ensemble perturbations). For more details, please refer to our recent publications:

• Miyoshi, T. and K. Kondo, 2013: A multi-scale localization approach to an ensemble Kalman filter. SOLA, 9, 170-173. doi:10.2151/sola.2013-038
• Kondo, K., T. Miyoshi and H. L. Tanaka, 2013: Parameter sensitivities of the dual-localization approach in the local ensemble transform Kalman filter SOLA, 9, 174-177. doi:10.2151/sola.2013-039

AIRS satellite data helped improve tropical cyclone forecasting

Our recent study indicated that using AIRS (Atmospheric Infrared Sounder) satellite data improved tropical cyclone forecasting significantly. AIRS, on board NASA's polar-orbiting spacecraft "Aqua", is an infrared sensor observing the Earth's radiation at different wavelengths and provides the horizontal and vertical structures of atmospheric temperature and moisture. The figure shows the observed track (black) of Typhoon Sinlaku (2008) and its forecasts initialized at 0600 UTC, September 12, 2008. Clearly, one of the forecasts shown by the red curve is closer to the black curve (observed). The red curve includes the AIRS data, while the blue curve does not.

It is widely known that satellite observations provide much information about atmospheric conditions. However, using satellite data effectively for weather forecasting is not a trivial task. Continuous efforts have been made to seek wiser use of satellite data for improving forecasts of significant weather. This study is a small step forward in this regard.

More details on this study can be found in our recent publication:
Miyoshi, T. and M. Kunii, 2012: Using AIRS retrievals in the WRF-LETKF system to improve regional numerical weather prediction. Tellus, 64A, 18408. doi:10.3402/tellusa.v64i0.18408

A paper on WRF-LETKF has been published

Miyoshi, T. and M. Kunii, 2012: The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations. Pure and Appl. Geophys., 169, 321-333. doi:10.1007/s00024-011-0373-4

A paper on observation error correlations has been accepted

Miyoshi, T., E. Kalnay, and H. Li, 2012: Estimating and including observation error correlations in data assimilation. Inv. Prob. Sci. Eng., in press.

A paper on lateral boundary perturbations has been published

Saito, K., H. Seko, M. Kunii, and T. Miyoshi, 2012: Effect of lateral boundary perturbations on the breeding method and the local ensemble transform Kalman filter for mesoscale ensemble prediction. Tellus, 64A, 11594. doi:10.3402/tellusa.v64i0.11594

A paper on observation impact estimates has been accepted

Kunii, M., T. Miyoshi, and E. Kalnay, 2011: Estimating impact of real observations in regional numerical weather prediction using an ensemble Kalman filter. Mon. Wea. Rev., in press.

A paper on variable localization and CO2 data assimilation has been published

Kang, J.-S., E. Kalnay, J. Liu, I. Fung, T. Miyoshi, and K. Ide, 2011: "Variable localization" in an Ensemble Kalman Filter: application to the carbon cycle data assimilation. J. Geophys. Res., 116, D09110. doi:10.1029/2010JD014673

A paper on adaptive inflation has been published

Miyoshi, T., 2011: The Gaussian Approach to Adaptive Covariance Inflation and Its Implementation with the Local Ensemble Transform Kalman Filter. Mon. Wea. Rev., 139, 1519-1535. doi:10.1175/2010MWR3570.1

A paper on tropical observation impacts estimated by ensemble spread has been published

Moteki, Q., K. Yoneyama, R. Shirooka, H. Kubota, K. Yasunaga, J. Suzuki, A. Seiki, N. Sato, T. Enomoto, T. Miyoshi, S. Yamane, 2011: The influence of observations propagated by convectively coupled equatorial waves. Quart. J. Roy. Meteor. Soc., 137, 641-655. doi:10.1002/qj.779

A paper on GPS-based moisture assimilation has been published

Seko, H., T. Miyoshi, Y. Shoji, and K. Saito, 2011: Data Assimilation Experiments of Precipitable Water Vapor using the LETKF System: Intense Rainfall Event over Japan 28 July 2008. Tellus, 63A, 402-414. doi:10.1111/j.1600-0870.2010.00508.x

A paper on balance and localization has been published

Greybush, S. J., E. Kalnay, T. Miyoshi, K. Ide, and B. R. Hunt, 2011: Balance and Ensemble Kalman Filter Localization Techniques. Mon. Wea. Rev., 139, 511-522. doi:10.1175/2010MWR3328.1

A paper on ozone data assimilation has been published

Sekiyama, T. T., M. Deushi, and T. Miyoshi, 2011: Operation-Oriented Ensemble Data Assimilation of Total Column Ozone. SOLA, 7, 41-44. doi:10.2151/sola.2011-011

A paper on WRF-LETKF has been accepted

Miyoshi, T. and M. Kunii, 2011: The local ensemble transform Kalman filter with the Weather Research and Forecasting model: experiments with real observations. Pure and Appl. Geophys.. doi:10.1007/s00024-011-0373-4