On the Effect of Sample and Topic Sizes for Argument Mining Datasets

The task of Argument Mining, that is extracting argumentative sentences for a specific topic from large document sources, is an inherently difficult task for machine learning models and humans alike, as large datasets are rare and recognition of argumentative sentences requires expert knowledge. The task becomes even more difficult when it also involves stance detection of retrieved arguments. Recent datasets for the task tend to grow evermore large and hence more costly. In this work, we inquire whether it is necessary for acceptable performance of argument mining to have datasets growing in size or, if not, how smaller datasets have to be composed for optimal performance. We also publish a newly created dataset for future benchmarking.