Kennesaw State student uses smartphone images and artificial intelligence to assess nutritional information about food

KENNESAW, Ga. | November 25, 2024

A simple “food selfie” can provide crucial nutritional data, and a smartphone application created by Kennesaw State University students could be the key to delivering that information as actionable insights to people monitoring their eating habits.

KSU graduate student Afnan Ahmed Crystal has been working with assistant professor of information technology Maria Valero and doctoral student El Arbi Belfarsi on The Nutrilyzer, an app that can extract nutritional data from a photo taken with a smartphone. Like Valero’s revolutionary GlucoCheck device, a non-invasive blood glucose monitoring system, the app is said to allow people to better track their daily eating habits through artificial intelligence (AI).

“We are trying to build an app to complement the GlucoCheck device, which will allow users to enter relevant information, take a simple photo with their smartphone and get an estimate of nutritional value using AI. This estimate will be used to analyze the impact on blood glucose in combination with the GlucoCheck device. We believe the app can empower people to make better decisions about nutrition.” said Crystal, who will graduate next month with her Master of Science in Computer Science.

An important part of the application is the use of AI. That’s where Belfarsi’s expertise in large language models (LLMs) comes into play. Originally working as a data analyst at a trading firm, he developed an interest in LLMs and sought to apply his expertise to health informatics systems. He saw the potential of AI to help people make better decisions about nutrition.

“Large language models are primarily designed for processing textual data,” says Belfarsi, who has a Ph.D. in computer science. “When dealing with images, they rely on additional layers, such as the Contrastive Language–Image Pretraining (CLIP) model, which is trained to match images with their textual descriptions. That is why when requesting nutritional data, we effectively use external computer vision models that complement the capabilities of the LLM.”

Valero credits the two students for combining their expertise and addressing potential pitfalls such as regional cuisines that could potentially cause ethnic bias in providing accurate information. The app is also made to adjust feedback based on portion sizes. It’s all part of the Internet of Things as a Service Research Group’s goal to help people live healthier lives through applied technology.

“Then we’re going to give the application to people, ask them to use our continuous glucose monitor and collect data,” Valero said. “With that data we can create a model for an individual prediction of how your glucose will behave. In addition, the app can prescribe other habits, such as exercise, to mitigate lifestyle and other factors.”

The project is not intended to replace the role of a dietitian or nutritionist, but to increase the human element with a useful app that everyone can use. Future versions of the app could consider exercise habits recorded through wearable technology such as Fitbits or Apple Watches.

“The main goal is to benchmark whether the LLMs can outperform a dietitian’s evaluation, or vice versa,” Crystal said. “The second is using AI for better decision-making about nutrition and improving people’s lives. We plan to integrate other wearable devices and push personalized data from these wearables to AI to improve responses to them.”

The Nutrilyzer is in early testing and may be launched this spring.

Story by Dave Shelles

Photo by Matt Yung

Related stories

A leader in innovative teaching and learning, Kennesaw State University offers bachelor’s, master’s and doctoral degrees to its more than 47,000 students. Kennesaw State is a member of the University System of Georgia with 11 academic colleges. The university’s vibrant campus culture, diverse population, strong global ties and entrepreneurial spirit attract students from across the country and the world. Kennesaw State is a Carnegie-designated doctoral research institution (R2), placing it in an elite group of only 7 percent of U.S. colleges and universities with R1 or R2 status. For more information, visit kennesaw.edu.