Threshold-Based Classification to Enhance Confidence in Open Set of Legal Texts.

Machine Learning has revolutionized the categorization of vast legal documents, minimizing costs and improving evaluations. However, conventional models struggle with unseen data categories in real-world scenarios, a challenge termed Open Set Classification. Our study tackles the issue faced by the Court of Justice in São Paulo, Brazil, to identify recurring lawsuit themes from texts, as manual sorting is inefficient. We introduce a method to enhance confidence in text classification using an open dataset by converting multiclass challenges into binary ones with four confidence tiers. By testing various techniques, we found that combining doc2vec with the Support Vector Machine classifier delivers trustworthy results and robust performance. Ultimately, our method offers an effective solution for classifying legal texts confronting Open Set Classification issues in the legal sector.