NLUBroker: A QoE-driven Broker System for Natural Language Understanding Services

Cloud-based Natural Language Understanding (NLU) services are becoming more popular with the development of artificial intelligence. More applications are integrated with cloud-based NLU services to enhance the way people communicate with machines. However, with NLU services provided by different companies powered by unrevealed AI technology, how to choose the best one is a problem for developers. Existing tools that can provide guidance to developers and make recommendations based on their needs are severely limited. This article comprehensively evaluates multiple state-of-the-art NLU services, and the results indicate that there is no absolute winner for different usage requirements. Motivated by this observation, we provide several insights and propose NLUBroker, a Quality of Experience-driven (QoE-driven) broker system, to select the proper service according to the environment. NLUBroker senses the client and service status and leverages a solution to the multi-armed bandit problem to conduct online learning, aiming to achieve maximum expected QoE. The performance of NLUBroker is evaluated in both simulation and real-world environments, and the evaluation results demonstrate that NLUBroker is an efficient solution for selecting NLU services. It is adaptive to changes in the environment, outperforms three baseline methods we evaluated and improves overall QoE up to 1.5x for the evaluated state-of-the-art NLU services.