On the Influence of an Iterative Affect Annotation Approach on Inter-Observer and Self-Observer Reliability.

Affect detection systems require reliable methods to annotate affective data. Typically, two or more observers independently annotate audio-visual affective data. This approach results in inter-observer reliabilities that can be categorized as fair (Cohen's kappas of approximately .40). In an alternative iterative approach, observers independently annotate small amounts of data, discuss their anno...