Harishankar Somasundaram · Writing · 18 August 2026
The Med AI clinical chat is a multi-step LangGraph agent that answers clinical questions in natural language over a live hospital estate: it classifies the question, plans which services to query, retrieves from them concurrently, drafts an answer with a locally-hosted LLM, and verifies every figure in that answer against the data it retrieved.
LangGraph compiles the clinical chat pipeline into an explicit state graph of six nodes — classify, plan, retrieve, assess, synthesize and verify — with two conditional edges. Each node takes the run state and returns a partial update, so planning, retrieval, generation and checking are separate, inspectable steps rather than one prompt. Two cycles are bounded by counters carried in the state: at most two planning rounds and one rewrite.
A regex classifier produces an intent as a hint, then a 3B planner model returns a JSON array of at most three tool calls, each with a stated reason. Tools whose required parameter is unknown are removed from the prompt, hallucinated tool names are dropped, and if the planner returns nothing for a question that plainly needs data, a per-intent table supplies a default source. The plan is a first-class object in the state, so it is logged, streamed to the UI and reviewable before it becomes an answer.
A verify node runs after the answer is drafted and checks every number in it against every number in the retrieved payloads. Figures that appear nowhere in the source either trigger one rewrite or are appended to the answer as an explicit unverified-figures warning. The check is arithmetic-free, so a percentage the model derived by dividing two source numbers is not accepted, and it is skipped entirely when nothing was retrieved, because a clinical-knowledge answer legitimately cites figures no hospital record contains.
No. The endpoint catalogue indexes GET operations only — currently 145 read endpoints across 18 FastAPI services. The estate also exposes more than a hundred write endpoints, including simulation resets, patient admission and GDPR erasure routes, and none of them are discoverable from chat. The chat is reachable from a public dashboard, so exposing a write has to be a deliberate allow-list rather than a side effect of discovery.
All models run locally through Ollama on a single RTX 4060 with 8 GB of VRAM. Every call names a task rather than a model, and the detected intent selects the task: llama3.2:3b handles intent detection, planning and prose over structured ML output, while deepseek-r1:8b handles open-ended clinical reasoning. Those two serve every request on this host; MedGemma 4B and OpenBioLLM 8B are wired into the routing table for note analysis and biomedical questions but are not pulled here. In production traces the 8B reasoning model's p95 latency is roughly twice the 3B model's, which is why only genuinely open-ended questions are routed to it.
Every run becomes one trace in a self-hosted Langfuse instance, session-grouped, with a span per graph node, a generation span for the streamed synthesis carrying the model name, and the verification verdict on the root span. Two things are easy to get wrong: an application that already installs a global OpenTelemetry provider will silently export the Langfuse callback spans to its own collector instead, and instrumenting only some of a service's entry points leaves the path the UI actually calls untraced.
Very little. In a representative traced run of 24.46 seconds, planning cost 0.88 s, retrieval 0.02 s, and classification, assessment and verification together under a tenth of a second — synthesis accounted for 23.55 s, or 96% of the request. Generating text dominates, so putting a planner and a verifier in front of it does not meaningfully slow the answer down.
No. The platform runs on MIMIC-IV derived records and a discrete-event hospital simulation. It is a demonstration of architecture rather than a clinical tool, and nothing it produces is clinical advice.