ENOS: a Holistic Framework for Conducting Scientific Evaluations of OpenStack

By massively adopting OpenStack for operating small to large private and public clouds, the industry has made it one of the largest running software project. Driven by an incredibly vibrant community, OpenStack has now overgrown the Linux kernel. However, with success comes an increased complexity; facing technical and scientific challenges, developers are in great difficulty when testing the impact of individual changes on the performance of such a large codebase, which will likely slow down the evolution of OpenStack. In the light of the difficulties the OpenStack community is facing, we claim that it is time for our scientific community to join the effort and get involved in the development and the evolution of OpenStack, as it has been once done for Linux. However, diving into complex software such as OpenStack is tedious: reliable tools are necessary to ease the efforts of our community and make science as collaborative as possible. In this spirit, we developed ENOS, an integrated framework that relies on container technologies for deploying and evaluating OpenStack on any testbed. ENOS allows researchers to easily express different configurations, enabling fine-grained investigations of OpenStack services. ENOS collects performance metrics at runtime and stores them for post-mortem analysis and sharing. The relevance of ENOS approach to reproducible research is illustrated by evaluating different OpenStack scenarios on the Grid’5000 testbed. Key-words: Performance, Reproducibility, OpenStack, Cloud, Visualization, Control-plane, Data-plane ENOS: un framework holistique pour l’évaluation d’OpenStack Résumé : Openstack, poussé par les industriels pour la gestion de leur infrastructure virtualisée et animé par une communauté très active, a dépassé le développment du noyau linux. En même temps que le succès, OpenStack connaît également une complexité grandissante. Les développeurs sont désormais en difficulté lorsqu’il faut vérifier l’impact de changements dans une large base de code. À la lumière de ces difficultés, nous pensons qu’il est temps pour la communauté scientifique de se joindre aux efforts et d’être impliquée dans le développement et les évolutions futures d’OpenStack comme cela a été le cas auparavant pour le noyau Linux. Dans cet esprit, nous avons développé ENOS, un framework qui s’appuie sur la technologie des conteneurs pour déployer et évaluer OpenStack sur différentes plateformes. Avec ENOS, les chercheurs peuvent facilement exprimer différentes configurations permettant une analyse fine des services constituant OpenStack. ENOS collecte automatiquement des métriques de performance et les stocke pour des analyses post-mortem. La pertinance d’ENOS dans le cadre d’expérimentation reproductible est illustré via différents scénario sur la plateform Grid’5000. Mots-clés : Performance, Reproductibilité, OpenStack, Coud, Visualisation, Control-plane, Data-plane ENOS: a Holistic Framework for Conducting Scientific Evaluations of OpenStack Ronan-Alexandre Cherrueau, Dimitri Pertin, Anthony Simonet, Adrien Lebre Inria, Mines Nantes, LINA Nantes, France Email: firstname.lastname@inria.fr , Matthieu Simonin Inria, IRISA Rennes, France Email: firstname.lastname@inria.fr By massively adopting OpenStack for operating small to large private and public clouds, the industry has made it one of the largest running software project. Driven by an incredibly vibrant community, OpenStack has now overgrown the Linux kernel. However, with success comes an increased complexity; facing technical and scientific challenges, developers are in great difficulty when testing the impact of individual changes on the performance of such a large codebase, which will likely slow down the evolution of OpenStack. In the light of the difficulties the OpenStack community is facing, we claim that it is time for our scientific community to join the effort and get involved in the development and the evolution of OpenStack, as it has been once done for Linux. However, diving into complex software such as OpenStack is tedious: reliable tools are necessary to ease the efforts of our community and make science as collaborative as possible. In this spirit, we developed ENOS, an integrated framework that relies on container technologies for deploying and evaluating OpenStack on any testbed. ENOS allows researchers to easily express different configurations, enabling fine-grained investigations of OpenStack services. ENOS collects performance metrics at runtime and stores them for post-mortem analysis and sharing. The relevance of ENOS approach to reproducible research is illustrated by evaluating different OpenStack scenarios on the Grid’5000 testbed.