Refine Neutrino Events Reconstruction with BEiT-3

Neutrino Events Reconstruction has always been crucial for IceCube Neutrino Observatory. In the Kaggle competition "IceCube -- Neutrinos in Deep Ice", many solutions use Transformer. We present ISeeCube, a pure Transformer model based on torchscale (the backbone of BEiT-3). Our model has the potential to reach state-of-the-art. By using torchscale, the lines of code drop sharply by about 80% and a lot of new methods can be tested by simply adjusting configs. Also, a relatively primitive loss function Mean Squared Error (MSE) can work really well. Since the model is simple enough, it also has the potential to be used for more purposes such as energy reconstruction, and many new methods such as combining it with GraphNeT can be tested more easily. The code and pretrained models are available at https://github.com/ChenLi2049/ISeeCube