Secure Image Inference Using Pairwise Activation Functions

Polynomial approximation has for the past few years been used to derive polynomials as an approximation to activation functions for use in image prediction or inference employing homomorphic encryption technique to induce data privacy and security. Most proposed works thus far have only been limited to deriving very few polynomials to use for these tasks. While the literature has considered forming new activation functions as pairwise multiplication of well-known activation functions, the design space is mostly unexplored. In some practical applications, there is usually a mix of activation functions used, so looking ahead, there is the need to explore into using other potential functions that can also improve performance whiles not relying on a few ones proposed such as ReLU and Swish. This paper explores the design space of such pairwise, multiplied activation functions and their application in homomorphic image inference or prediction using the widely popular MNIST and CIFAR-10 benchmark datasets. Moreover, we analyzed corresponding curve fitting parameters (range and degree), homomorphic-friendly pooling methods, and optimization methods in the ciphertext domain to avoid incurring huge computation costs but not compromising accuracy. Results show new activation function combinations yielding similar or better results in ciphertext as compared to the ones in plaintext.