Verification of integrity of deployed deep learning models using Bayesian Optimization

Artificial Intelligence (AI) is hugely driven by machine learning (ML). Popular ML algorithms like deep neural networks (DNN) are used in wide variety of applications. Due to excellent predictive capabilities, these models are used in many realworld applications. To accommodate the huge service demands from these models, they are hosted on cloud platforms. Hosting a model in cloud raises security concerns. Malicious third party can modify the model when in transit from local system to cloud platform or after being uploaded in the cloud. We propose Bayesian Compromise Detection (BCD) algorithm to detect such model compromises by generating sensitive samples. Finding a sensitive sample is essentially an optimization problem that aims to maximize the prediction differences between the original and the compromised model. The optimization problem is challenging as (1) a cloud customer can only have a blackbox access to the compromised model (2) sensitive sample has to be searched in the distribution of the training inputs, which is most likely a high-dimensional space and (3) the optimization problem is non-convex. To tackle these obstacles, we use Variational Autoencoder (VAE) to map high-dimensional data to a non-linear low-dimensional space and then use Bayesian Optimization (BO) to find the globally optimum sensitive sample. Proposed algorithm generates a sensitive sample that can detect model compromise without incurring much cost by multiple queries. Experiments with multiple datasets demonstrate the effectiveness of our proposed method. Our method outperforms the state-of-the-art method in terms of detection rate of the compromised models on cloud platforms.