Fine-tuning GPT-3 for legal rule classification

In this paper, we propose a Legal Rule Classification (LRC) task using one of the most discussed language model in the field of Artificial Intelligence, namely GPT-3, a generative pretrained language model. We train and test the proposed LRC task on the GDPR encoded in LegalDocML (Palmirani and Vitali, 2011) and LegalRuleML (Athan et al., 2013), two widely used XML standards for the legal domain. We use the LegalDocML and LegalRuleML annotations provided in Robaldo et al. (2020) to fine-tuned GPT-3. While showing the ability of large language models (LLMs) to easily learn to classify legal and deontic rules even on small amount of data, we show that GPT-3 can significantly outperform previous experiments on the same task. Our work focused on a multiclass task, showing that GPT-3 is capable to recognize the difference between obligation rules, permission rules and constitutive rules with performances that overcome previous scores in LRC.