Collaborative quality filtering: establishing consensus or recovering ground truth?

We present a algorithm based on factor analysis for performing collaborative quality filtering (CQF). Unlike previous approaches to CQF, which estimate the consensus opinion of a group of reviewers, our algorithm uses a generative model of the review process to estimate the latent intrinsic quality of the items under reviews. We run several tests that demonstrate that consensus and intrinsic quality are, in fact different and unrelated aspects of quality. These results suggest that asymptotic consensus, which purports to model peer review, is, in fact, not recovering the ground truth quality of reviewed items.