Information Theoretic Probing with Minimum Description Length

EMNLP 2020, pp. 183-196, 2020.

Cited by: 0|Views19
Weibo:
We propose information-theoretic probing which measures minimum description length of labels given representations

Abstract:

To measure how well pretrained representations encode some linguistic property, it is common to use accuracy of a probe, i.e. a classifier trained to predict the property from the representations. Despite widespread adoption of probes, differences in their accuracy fail to adequately reflect differences in representations. For example, th...More

Code:

Data:

0
Full Text
Bibtex
Weibo
Your rating :
0

 

Tags
Comments