Computer Vision for Dietary Assessment

Automated visual recognition of food from smartphone cameras could be a powerful tool for assisting people to track their eating behaviors. Existing work in computer vision has focused on coarse-grained food classification, typically on idealized food images collected from the web, which may not reflect the challenges of real-world foods or photos. Despite advancements in computer vision over the last few years, error rates in these food recognition studies are quite high compared to human observers. We argue that we need to rethink how computer vision and AI can automate food logging, such as understanding the types of relationships humans have with foods, or creating semi-automatic tools that could complement dietitians instead of replacing them.