Gas limit aware mutation testing of smart contracts at scale

The blockchain is a key technology that has been adopted in many application areas to increase security and reliability and to avoid the need for a central trusted authority. One of its essential underlying foundations are smart contracts, which are executable programs for managing data or assets on the blockchain. It is crucial that smart contracts are tested thoroughly due to their immutable nature and since even small bugs can lead to huge monetary losses. However, it is not enough to just test smart contracts, it is also important to ensure the quality and completeness of the tests. Hence, we introduce new smart contract specific mutation operators as well as a novel killing condition that is able to detect a deviation in the gas consumptions, i.e., in the monetary value that is required to perform transactions. Moreover, we establish a baseline for mutation testing of smart contracts by applying our method to a replay test suite and by testing about a thousand contracts.