KOLE: breaking the scalability barrier for managing far edge nodes in cloud

ABSTRACTIn edge computing, the trend is moving towards leveraging cloud native technologies and platforms such as con-tainerization and Kubernetes to manage edge applications to improve operation efficiency. Unfortunately, the supported number of nodes per cluster is only several thousand in Kubernetes which is much less than what typical far edge use cases require. In this paper, we propose KOLE, a framework that extends the upstream Kubernetes for supporting a large number of far edge nodes. It replaces the existing apiserver-to-node communication mechanism in Kubernetes with a MQTT messaging system. The MQTT broker completely offloads the overhead of keeping numerous HTTP connections for nodes in the apiserver. In KOLE, we avoid creating numerous individual objects in the apiserver by maintaining them in a cloud state cache. The cache is snapshotted periodically for disaster recovery. Overall, KOLE achieves outstanding scalability by sacrificing the manageability of having individual objects, which we believe is a reasonable trade-off for far edge use cases. The experiment results show that KOLE is scalable and can support up to one million nodes.