Beyond Rating Scales: With Targeted Evaluation, Language Models are Poised for Psychological Assessment

•Artificial intelligence has been undergoing a purported “paradigm shift” initiated by new machine learning models, large language models.•We review recent empirical evaluations of AI-based language assessments and present a case for the technology of large language models, that are used for chatGPT and BERT, to be poised for changing standardized psychological assessment.•While potential applications for automated therapy are beginning to be studied on the heels of chatGPT's success, here we present evidence that suggests, with thorough validation of targeted deployment scenarios, that AI's newest technology can move mental health assessment away from rating scales and to instead use how people naturally communicate, in language.