Towards Enhanced Human Activity Recognition through Natural Language Generation and Pose Estimation
CoRR(2023)
摘要
Vision-based human activity recognition (HAR) has made substantial progress
in recognizing predefined gestures but lacks adaptability for emerging
activities. This paper introduces a paradigm shift by harnessing generative
modeling and large language models (LLMs) to enhance vision-based HAR. We
propose utilizing LLMs to generate descriptive textual representations of
activities using pose keypoints as an intermediate representation.
Incorporating pose keypoints adds contextual depth to the recognition process,
allowing for sequences of vectors resembling text chunks, compatible with LLMs.
This innovative fusion of computer vision and natural language processing holds
significant potential for revolutionizing activity recognition. A proof of
concept study on a Kinetics700 dataset subset validates the approach's
efficacy, highlighting improved accuracy and interpretability. Future
implications encompass enhanced accuracy, novel research avenues, model
generalization, and ethical considerations for transparency. This framework has
real-world applications, including personalized gym workout feedback and
nuanced sports training insights. By connecting visual cues to interpretable
textual descriptions, the proposed framework advances HAR accuracy and
applicability, shaping the landscape of pervasive computing and activity
recognition research. As this approach evolves, it promises a more insightful
understanding of human activities across diverse contexts, marking a
significant step towards a better world.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要