Fine-tuning GPT-J for text generation tasks in the Slovak language

This paper provides a review of the GPT-J demonstration with a specific focus on its fine-tuning capabilities for text generation in Slovak. Through reasoning, we explore the model’s fine-tuning in tasks such as text generation. We compare the results between different sizes of datasets and different sizes of pre-trained models. The results clearly show that the size of the dataset has an impact on the quality of the resulting model.