Hosting Astronomical Data in Sharded SQL Databases

At Leibniz-Institute for Astrophysics Potsdam (AIP) we host and publish terabytes of cosmological simulation and observational data. A dedicated web application based on the Daiquiri framework allows scientists all around the world to run SQL queries which include specific astrophysical functions and get their desired data in reasonable time. The setup is based on sharded MariaDB database nodes, orchestrated by a head node which runs the MariaDB Spider engine. ParallelQuery (PaQu) reformulates the SQL queries for the use in distributed environments. The web framework Daiquiri offers a dedicated web interface for each of the hosted scientific databases. All software we develop is published under an Open Source license.