To address this problem, University of Waterloo researchers are taking aim at malnutrition in aging populations with the development of an AI-powered application that can accurately measure people’s caloric and nutrient intake simply by "watching" what they eat.
Unlike conventional nutrition apps that typically rely on the user to input information to make calculations and recommendations, the AI-powered app does all the work. The user doesn’t need to upload photos of their food, search a database for specific items or accurately measure and input how much of each item they’re eating.
To use the app, you simply sit down at your table with a plate of food and a video camera mounted in a hardware device automatically detects when you start eating and records every bite that goes into your mouth. Behind the scenes, a visual language model (VLM) automatically analyzes the video, frame by frame, considering the utensil you’re using and the different stages of eating, including opening your mouth and chewing. Meta AI’s Segment Anything Model (SAM) keeps the nutrient analysis accurate by honing in on exactly what you’re eating and how much.
"The more an app requires user input, the more likely it is that people won’t continue using it long term," says Dr. Yuhao Chen, a research assistant professor in the Department of Systems Design Engineering’s Vision and Image Processing (VIP) Lab. "We want to support ongoing nutrition monitoring, so we’re using AI to automate processes and place fewer requirements on the user."
From presumption to precision
The new system has been highly effective at analyzing the calories and nutrients of foods consumed using a spoon. The research is now expanding in scope to include more challenging utensils, like forks, chopsticks and even hands.The team also plans to go beyond analyzing food to incorporate other substances impacting a senior’s health and well-being. "The vision is to analyze whatever a person ingests, including supplements and drugs," Chen says.
No matter how the technology evolves, the aim remains to provide accurate caloric and nutrient tracking to older adults and their caretakers, make recommendations to fill identified nutritional gaps, and to do this in a way that is simple and user-friendly.
"Some people aren’t comfortable using technology, so we want to make our application very easy to use," says Chris Czarnecki (MASc ’24), a recent master’s graduate who worked on the project. "It could take the form of a smartphone app that uses the device’s built-in camera, or even wearable glasses.
"The goal is to help aging populations achieve optimal nutrition. Commercializing a solution that gives this growing demographic the tools to thrive while living independently in their own homes would be a huge success."
This AI-powered nutrition tracking system is an example of Waterloo’s innovative research on the determinants of health, including healthy aging. With the number of citizens aged 65 and older predicted to double over the next 30 years, widespread malnutrition in older adults has the potential to cripple long-term care and health-care systems.
It’s an initiative focused on improving the quality of life of older adults and their caregivers through innovation for safe and healthy aging.
