Crowdsourcing Lightweight Pyramids for Manual Summary Evaluation
arXiv: Computation and Language, 2019.
Conducting a manual evaluation is considered an essential part of summary evaluation methodology. Traditionally, the Pyramid protocol, which exhaustively compares system summaries to references, has been perceived as very reliable, providing objective scores. Yet, due to the high cost of the Pyramid method and the required expertise, rese...More
PPT (Upload PPT)