AIM: Automated Input Set Minimization for Metamorphic Security Testing

For Web systems, which are accessible to any machine connected to internet, security is a critical concern. Although security testing can be automated by generating crafted inputs as an attacker would do, solutions to automate the test oracle, i.e., distinguishing correct from incorrect outputs for a given input, remain preliminary. Specifically, previous work has demonstrated the potential of metamorphic testing; indeed, security failures can be determined by metamorphic relations that turn valid inputs into malicious inputs and compare their outputs. However, without further guidance, metamorphic relations should be executed on a very large set of valid inputs, which is time consuming and makes metamorphic testing impractical. Hence, in this study, we propose AIM, an approach that automatically selects inputs to reduce testing costs while preserving vulnerability detection capabilities. AIM includes a clustering-based black box approach, identifying similar inputs based on their security properties. It also presents a novel genetic algorithm able to efficiently select diverse inputs while minimizing their total cost. Further, it contains a problem reduction component to reduce the search space and speed up the minimization process. We evaluated the effectiveness of AIM on two well-known web systems, Jenkins and Joomla. We compared AIM's results with four baselines in security testing. Overall, AIM reduced MRs execution time by 84 percent for Jenkins and 82 percent for Joomla while preserving full vulnerability detection. Furthermore, AIM outperformed all the considered baselines regarding vulnerability coverage. Although it has been tuned to work with Web system inputs, AIM could be applied to minimize metamorphic testing cost in other contexts.