The High Data Rate Processing and Analysis Initiative of the Helmholtz Association in Germany

The Helmholtz Association is the largest scientific organization in Germany. It operates all major German research infrastructures involved in research with photons, neutrons, and ions. These are DESY in Hamburg; the Karlsruhe Institute of Technology (KIT); the Research Centre Julich (FZJ); the Helmholtz Centres in Geesthacht (HZG), Berlin (HZB), and Dresden-Rossendorf (HZDR); and the GSI Centre for research with heavy ions in Darmstadt. In common, all these centers are facing similar challenges related to dramatically increasing data rates and volumes generated with more and more powerful radiation sources together with larger and faster detectors. On the other hand, each center has its own specific portfolio of long-lasting technical expertise in areas like data analysis, information technology, or hardware development. Therefore, it was obvious to address the challenges by acting in concert. This was the main motivation in 2010 for the launch of a joint project among the partners called the “High Data ...