Exploring New Monitoring and Analysis Capabilities on Cray’s Software Preview System

Cray, NCSA, and Sandia staff and engineers are collaborating to jointly investigate and provide new insights on the monitoring aspects of Cray’s recently released “Software Preview System.” In the preview system, Cray has implemented the LDMS framework within the monitoring infrastructure. In this work, we extend that implementation and leverage the Cray infrastructure to include new monitoring capabilities suitable for addtional node-level and application monitoring. We use the Cray-provided telemetry bus for transport and consumption of the new metric data. We explore scale and performance considerations. We provide details on the issues impacting or facilitating implementation of these functionalities within Cray’s new, container-based services system. In our implementation, we adhere to Cray’s design philosophy which is intended to ensure the reliability and availability of the Craycollected metrics and system services.