Towards a cloud performance research repository

In this paper, we describe our goals and initial efforts to create a community wide infrastructure for empirical research on cloud systems performance modeling. Performance analysis and prediction has long been an active research area in software engineering. Recent work in this area has focused on cloud-deployed applications that are characteristic of an ever growing percentage of modern software systems. The drivers for cloud deployment are improved quality attributes such as performance and scalability, access to advanced cloud-based technologies, and reduced costs. In terms of research, efforts include optimizing resource utilization for cloud systems under various workloads, understanding the influence on performance of various cloud-based infrastructure components, and architecture design tradeoffs. To support these efforts and energize the creation of new models and methods, our goal is to build a repository that stores both raw and analyzed performance results from a collection of benchmarks characterizing different types of cloud applications. This data can be used by researchers to explore, calibrate and validate higher fidelity performance models and identify new research directions. Results from benchmarks can also help architects understand the effects of their architectural decisions and cloud services selections and configurations on application quality attributes.