Predicting Mouse Click Position Using Long Short-Term Memory Model Trained by Joint Loss Function

BSTRACT Knowing where users might click in advance can potentially improve the efficiency of user interaction in desktop user interfaces. In this paper, we propose a machine learning approach to predict mouse click location. Our model, which is LSTM (long short-term memory)-based and trained by joint supervision, can predict the rectangular region of mouse click with feeding mouse trajectories on the fly. Experiment results show that our model can achieve a result of a predicted rectangle area of 58 × 79 pixels with 92% accuracy, and reduce prediction error when compared with other state-of-the-art prediction methods using a multi-user dataset.