MagPySV: A Python Package for Processing and Denoising Geomagnetic Observatory Data

Measurements obtained at ground-based observatories are crucial to understanding the geomagnetic field and its secular variation (SV). However, current data processing methods rely on piecemeal closed-source codes or are performed on an ad hoc basis, hampering efforts to reproduce data sets underlying published results. We present MagPySV, an open-source Python package designed to provide a consistent and automated means of generating high-resolution SV data sets from hourly means distributed by the Edinburgh World Data Centre. It applies corrections for documented baseline changes, and optionally, data may be excluded using the ap index, which removes effects from documented high solar activity periods such as geomagnetic storms. Robust statistics are used to identify and remove outliers. Developing existing denoising methods, we use principal component analysis of the covariance matrix of residuals between observed SV and that predicted by a global field model to remove a proxy for external field contamination from observations. This method creates a single covariance matrix for all observatories of interest combined and applies the denoising to all locations simultaneously, resulting in cleaner time series of the internally generated SV. In our case studies, we present cleaned data in two geographic regions: monthly first differences are used to investigate geomagnetic jerk morphology in Europe, an area previously well-studied at lower resolution, and annual differences are investigated for northern high latitude regions, which are often neglected due to their high noise content. MagPySV may be run on the command line or within an interactive Jupyter notebook; two notebooks reproducing the case studies are supplied.