Researching an AI dietician with National Healthcare Group Polyclinics

I worked on a project with National Healthcare Group Polyclinics that looked at the best way to implement an AI dietician. The first prototypes were honestly quite exciting to see. They could identify food, talk in a friendly way, and give an answer that looked impressive on the surface.

But they also could not do very much beyond that. Once we looked past the demo effect, it became clear that a useful AI dietician needed more grounding. It needed food nutrition data, restaurant context, semantic search, and a way to avoid giving confident but unsafe advice.

The project expanded into a broader prototype with several data sources and services:

• Google Maps API for restaurant and nearby food context.
• HPB nutrition data to ground food and nutrient information.
• AWS services and Milvus for semantic search over relevant food nutrition data.
• Guardrails to manage what the assistant should answer, clarify, or avoid.

One interesting observation came from the image processing pipeline. Normal object detection may struggle to label a small sauce dish beside satay correctly, because the sauce itself can be visually ambiguous. A more useful system should combine visual cues with context and semantic association: if the main food is satay and there is a small sauce dish beside it, it is more likely to be satay sauce than a random brown condiment.

That is the kind of detail that made the project feel different from a normal chatbot. Food advice seems simple until it is connected to health, chronic conditions, user behaviour, and local eating patterns. In that setting, being useful sometimes means refusing, narrowing, or asking for clarification. I think that was the biggest lesson: safety and usefulness have to be designed together.

Footnote: Ported over from my personal blog. Initially posted on 10 December 2025.