Human-in-the-Loop Automatic Data Migration for a Large Research Computing Data Center

Most HPC centers face a lack of expertise of data center migrations, as it's a rare event that only a small portion of HPC professionals experience more than once in their entire professional careers. This paper presents how the Georgia Institute of Technology (Georgia Tech) Partnership for an Advanced Computing Environment (PACE) team employed automation to migrate research computing data from the old Rich computing center (Rich) to the new Coda data center (Coda) in 2020. PACE successfully migrated 1844 TB of data for 3550 users without loss of user data. PACE implemented a `human-in-the-loop' automatic workflow to facilitate the migration, interleaving automated scripts with human-driven reviews, significantly reducing staff time commitment while ensuring the integrity and accuracy of data migrations. PACE deployed a cached data movement strategy which reduced the migration downtime significantly. We share our one-year migration journey for the benefit of the HPC community.