A synthetic study on a coupled Kalman smoother/filter technique to a data assimilation algorithm

Our previous work revealed the potential of integrating Kalman smoother techniques within a data assimilation framework [1], specifically in the Estimating Model Parameters Ionospheric Reverse Engineering (EMPIRE) algorithm [2,3,4,5], for a storm event occurring from March 16th to 18th, 2015. However, the improvements remained ambiguous due to the absence of a validation source. To address this, we proposed a methodology leveraging SAMI3 model outputs as ground truth data, incorporating error statistics reconstruction fused with representation error sources within EMPIRE [6]. Subsequently, a comprehensive global error analysis was conducted by comparing EMPIRE outputs against the SAMI3 "truth."
In the current study, we continue this investigation by adopting a similar methodological approach, utilizing SAMI3 as the truth information. We aim to compare EMPIRE's performance between its primary setup (Kalman filter-based only), and the new setup incorporating a coupled Kalman filter/smoother technique. This investigation seeks to illustrate how the Kalman smoother can enhance the state estimations within EMPIRE, potentially benefiting other data assimilation (DA) algorithms as well.
Our preliminary findings reveal notable enhancements in ion drift estimations, particularly at high latitudes exceeding +/- 60 degrees, with the adoption of the coupled Kalman filter/smoother technique. Specifically, in the field-perpendicular zonal direction, the zonally averaged or latitude-dependent mean error decreases from approximately 200 m/s to around 100 m/s during storm events. Similarly, in the field-perpendicular meridional direction, the zonally averaged mean error shows a reduction to 50 m/s compared to the roughly 100 m/s observed in the primary EMPIRE setup.
These outcomes underscore the potential efficacy of integrating Kalman smoother methodologies within the EMPIRE algorithm, showcasing improved accuracy in ionospheric state estimations, especially under dynamic storm conditions and at high latitudes. This advancement holds promise not only for EMPIRE's performance but also the potentials for enhancing state estimation capabilities across various data assimilation algorithms.

Reference
[1] Hu, J., Rubio, A. L., Chartier, A., Bust, G. S., & Datta-Barua, S. (2023) Filtering/smoothing coupled technique implementation to the data assimilation algorithm [Poster]. Coupling, Energetics and Dynamics of Atmospheric Regions Program (CEDAR), San Diego. https://cedarscience.org/poster-2023/filteringsmoothing-coupled-techniq…
[2] Datta‐Barua, S., Bust, G. S., Crowley, G., & Curtis, N. (2009). Neutral wind estimation from 4‐D ionospheric electron density images. Journal of Geophysical Research: Space Physics, 114(A6).
[3] Miladinovich, D. S., Datta-Barua, S., Bust, G. S., & Makela, J. J. (2016). Assimilation of thermospheric measurements for ionosphere-thermosphere state estimation. Radio Science, 51(12), 1818-1837.
[4] Miladinovich, D. S., Datta‐Barua, S., López Rubio, A., Zhang, S. R., & Bust, G. S. (2020). Assimilation of GNSS Measurements for Estimation of High‐Latitude Convection Processes. Space Weather, 18(8), e2019SW002409.
[5] López Rubio, A., & Datta‐Barua, S. (2022). Vector Spherical Harmonics for Data‐Assimilative Neutral Wind Estimation. Space Weather, 20(8), e2022SW003052.
[6] Hu, J., Rubio, A. L., Chartier, A., McDonald, S., & Datta-Barua, S. (2024). Quantification of representation error in the neutral winds and ion drifts using data assimilation. American Geophysical Union, Space Weather. (Accepted March 2024)