Binary classification threatens the validity of cognitive impairment detection.

Our results show that binary classification can lead to a situation where two highly similar batteries fail to identify the same individuals as impaired. Additional simulations suggest that within-person agreement between assessment methods using binary classification is inherently low. Modern statistical tools may help to improve validity of impairment detection. (PsycInfo Database Record (c) 2023 APA, all rights reserved).