Smartphone-based Handwritten Signature Verification using Acoustic Signals

AbstractHandwritten signature verification techniques, which can facilitate user authentication and enable secure information exchange, are still important in property safety. However, on-line automatic handwritten signature verification usually requires dynamic handwritten patterns captured by a special device, such as a sensor-instrumented pen, a tablet or a smartwatch on the dominant hand. This paper presents SonarSign, an on-line handwritten signature verification system based on inaudible acoustic signals. The key insight is to use acoustic signals to capture the dynamic handwritten signature patterns for verification. Particularly, SonarSign exploits the built-in speakers and microphones of smartphones to transmit a specially designed training sequence and record the corresponding echo for channel impulse response (CIR) estimation, respectively. Based on the sensitivity of CIR to the tiny surrounding environment changes including handwritten signature actions, SonarSign designs an attentional multi-modal Siamese network for end-to-end signatures verification. First, multi-modal CIR streams are fused to extract representative signature pattern features from spatio-temporal dimensions. Then an attentional Siamese network is elaborated to verify whether the given two signatures are from the same signatory. Extensive experiments in real-world scenarios show that SonarSign can achieve accurate and robust signatures verification with an AUC (Area Under ROC (Receiver Operating Characteristic) Curve) of 98.02% and an EER (Equal Error Rate) of 5.79% for unseen users.