Evaluation of Lexical Acquisition Algorithms

Since there is a continuously growing amount of lexical acquisition algorithms for all kinds of lexical knowledge, be it morphologic, syntactic, semantic, domain specific or other information, it is necessary to be able to compare their quality. Furthermore at the same time it is necessary to provide an evaluation method that is easy to grasp and as uniform as possible so that it can be applied to all kinds of algorithms and provide comparable and most importantly reproducible results. We propose an extension of the gold standard evaluation method by generalizing most lexical knowledge to binary relations and then give an evaluation framework based on precision and recall. To show the effectiveness of this approach we compare the well-known mutual information measure with the dice coefficient. Information Maßes und des Dice Koeffizienten dar.