Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation

Ori Shapira
Ori Shapira
Hadar Ronen
Hadar Ronen
Ramakanth Pasunuru
Ramakanth Pasunuru

arXiv: Computation and Language, 2019.

Cited by: 3|Bibtex|Views38|Links
EI

Abstract:

Conducting a manual evaluation is considered an essential part of summary evaluation methodology. Traditionally, the Pyramid protocol, which exhaustively compares system summaries to references, has been perceived as very reliable, providing objective scores. Yet, due to the high cost of the Pyramid method and the required expertise, rese...More

Code:

Data:

Your rating :
0

 

Tags
Comments