It Takes More Than One Agent
It’s easy to picture “the AI” behind a chat interface as one thing. A single model, reading a question and writing an answer, the same way a person might picture one clinician behind a phone line rather than a whole department. The conversation certainly feels continuous enough to support that picture: one thread, one voice, no obvious seam where one capability ends and another begins.
Does one agent do everything?
It doesn’t, and the first post in this series already introduced part of the reason why: a router at the front of the example application’s chat hands a question to one of several specialist sub-agents, each with a narrower job than “answer anything about this patient.” That’s worth taking further than a passing introduction. A workflow made of narrowly scoped specialists, each following its own fixed sequence of steps, checked by something other than itself before anything reaches a person, is a different kind of thing to reason about than one general-purpose assistant, and it’s worth looking at as an ensemble rather than through any single member of the cast.
What is an agent anyway?
It’s worth pausing on what “an agent” actually means here, since the word gets used loosely. For the purposes of this series, an agent is an autonomous process built around a large language model, given a goal rather than a script, and able to call on tools, retrieval and other capabilities to work out how to fulfil it, rather than simply answering from what it already knows. Building one from scratch, prompt by prompt, is possible, but it’s increasingly common to build agent workflows on top of a dedicated software development kit that handles the mechanics, running steps, managing state, wiring in tools, so the application code can focus on what each agent is actually meant to do. The example application is built on Google’s Agent Development Kit (ADK), alongside others such as LangGraph and Microsoft’s AutoGen. To be clear, we are not saying one is better than another or advocating the use of any the specific Agent SDK. As always, other software development kits are available, and it’s well worth doing your own research before settling on one.
It’s also worth naming the mood “agent” tends to arrive in at the moment. Reports of AI agents behaving unpredictably, or going further than intended, have become a fairly regular feature of the news over the past couple of months, and it would be easy to read “autonomous process” in the definition above and picture something considerably more dramatic than what actually sits behind the example application’s chat interface. How much weight any individual report deserves is worth forming a view on for yourself rather than taking from a summary post on our company blog. What’s less in question is that a healthcare setting is the wrong place to be casual about it either way, which is the more useful question underneath “can an agent be trusted with this”: not a claim that it won’t misbehave, but a design that narrows what it’s able to do, checks what it produces, and can account for every step it took, which is exactly what the rest of this series, and this post in particular, has been describing. An agent that’s tightly scoped, audited and validated is perhaps a considerably smaller thing to worry about than “autonomous process” makes it sound.
This article explores the idea of a workflow of such agents using the same example application as the rest of the series. As before, it isn’t a recommendation for how an agent workflow should be built. It’s a set of considerations, drawn from working with the example application, that architects, engineers and interested clinicians may find useful when testing their own designs.
TL;DR
🎭 A chat interface can hide an entire cast, not one model wearing different hats.
🧭 Something still has to decide who’s asked, and check whether they’re allowed to ask it.
🪪 Not every member of the cast needs the same identity. Reach is granted deliberately, not by default.
🧩 Every specialist follows a fixed pipeline. The model’s freedom sits inside a stage, not between them.
🔁 Nothing reaches the page without a second opinion checking it first.
🖊️ Reading and writing aren’t trusted the same amount, and only one agent gets to write.
The full cast
Some of this cast has already appeared elsewhere in the series. It’s worth gathering the whole ensemble in one place before looking at how they work together.
Patient Record Explorer (Explorer)
Met in the first post, answers open-ended questions by searching the semantic layer described in the fourth and fifth posts.
Patient Record Enquirer (Enquirer)
Also met in the first post, answers a fixed set of well-defined questions from a library of pre-written FHIR searches, with no language model involved once the right template has been found.
Patient Record Appointment Scheduler (Scheduler)
Met in the first and second posts, finds and books appointment slots.
Patient Record Summariser (Summariser)
Whose output appeared in the first post’s discussion of generated content, produces a structured overview of the whole record.
Patient Record Document Filer (Document Filer)
Also introduced in the first and second posts, handles an uploaded clinical document.
Greeter
Handles nothing but the opening of a conversation.
General
Fields anything that doesn’t fit the others, and is explicitly told not to diagnose and not to claim it can look anything up.
None of these agents can do what the others do. The Enquirer can’t search freely the way the Explorer can. The Explorer can’t book an appointment. General can’t touch the record at all. That narrowness is doing real work. A single, general-purpose assistant with access to everything has to be trusted, in one lump, with every capability the application offers. A cast of specialists, each scoped to one job, only has to be trusted with the one job it actually has, and the boundary between jobs becomes something you can point at, rather than something you have to take on faith about a single model’s judgement.
Somebody still has to decide who answers
Splitting the work across specialists only helps if something reliably sends each question to the right one, so a small sequence runs ahead of all of them. A check for whether the message is simply confirming a choice the system itself just offered, handled without a model call at all. A check for a greeting. Then an actual classification: a language model, run at a low temperature with no tools available to it, whose only job is to say which of the cast a question calls for.
That classification isn’t the end of it. An ordinary piece of code takes the classifier’s answer and decides what actually happens next, substituting a fixed refusal for whichever specialist the classifier named if the person asking doesn’t have clinical access, the same role check the second post described being applied to pages in the application. It’s worth keeping those two jobs distinct even though they run back to back: the classifier answers a question about meaning, what is this person asking, and the router answers a question about permission, given what they’re asking and who they are, what’s allowed to happen next. Neither one has to do the other’s job.
flowchart LR
A("Question arrives") --> B("Classifier and role-checked router")
B --> C("Greeter")
B --> D("General")
B --> E("Explorer")
B --> F("Enquirer")
B --> G("Summariser")
B --> H("Appointment Scheduler")
classDef base fill:#437c9f,stroke:#2f5f79,color:#ffffff,stroke-width:1px;
classDef highlight fill:#6dc067,stroke:#437c9f,color:#1f2d1f,stroke-width:1.2px;
class A,C,D,E,F,G,H base;
class B highlight;
linkStyle default stroke:#437c9f,stroke-width:1.2px;
Not every specialist needs the same reach
The cast doesn’t all carry the same identity, either, and that’s a further design decision rather than an oversight. The example application gives the Explorer, the Enquirer, the Appointment Scheduler, the Summariser and the Document Filer each a fixed identity of their own, the Device references the first post described appearing in audit trails and provenance records. Greeter, General and the classifier itself have none, because none of them ever touch the FHIR record: there’s nothing for an identity to be attached to.
That’s a small detail with a wider point behind it. Identity, in this workflow, is granted to the agents that need to be accountable for something, not handed out uniformly across the whole cast as a matter of course. An agent that can’t read or write anything doesn’t need to carry a badge for doors it never tries to open. Deciding which members of an ensemble need an identity at all, rather than assuming every agent does, or that none of them do, is worth doing as deliberately as the first post’s question of how granular that identity should be.
Each pipeline is fixed, even when the model isn’t
It’s tempting to picture each specialist, once chosen, as a single freewheeling conversation with a language model: the question goes in, the agent works out what to do, an answer comes out. That isn’t quite what happens for any of them. Every specialist in the example application runs a fixed sequence of stages, and the model’s freedom is scoped to what happens within a stage, not to which stages run or in what order.
The Explorer always runs the same shape: a planner works out a patient-scoped, rewritten search from the question, a fixed retrieval step runs and is audited exactly as the first post described, and a loop of up to five attempts alternates between a synthesiser writing an answer and a validator checking it. The Enquirer is narrower still: a template matcher, an executor applying the role and patient checks from the second post, and a fixed renderer, with no further model calls once the template’s been chosen. The Summariser follows a similar shape to the Explorer, with an added step where a resolver model narrows a wider pool of retrieved candidates down to the ones a question was actually about before synthesis begins. The Appointment Scheduler splits its own work across two separate model calls with two separate sets of tools, which the next section picks up.
The Document Filer runs a pipeline of its own, and it’s worth walking through in a little more detail, because filing a document is a slightly different kind of trust decision to answering a question. A format check runs first, and text is extracted from whatever was uploaded. Before anything is treated as part of the record, a language model scores how confident it is that the document actually looks like a piece of clinical content at all, checked against a high threshold, so an upload that isn’t recognisably a care record (e.g. a stray photograph of a cat), rather than a letter, say, doesn’t proceed any further. A second, separate check reads the extracted text specifically for anything that looks like an attempt to manipulate the agent itself, a document engineered to steer the summarising step that follows, rather than one that simply describes a patient. Only once both checks pass does the pipeline move on to summarising and coding the document. Either check failing blocks the upload outright, rather than filing something uncertain and hoping a later stage catches it.
Laid out together, what stands out isn’t any single stage. It’s that the order is fixed by code for every one of these agents, not chosen by a model turn to turn. A model deciding what a chunk of retrieved text means is a reasonable thing to trust it with. A model deciding whether it feels like checking its own citations, or a document’s authenticity, this time is not, and a fixed pipeline, repeated across the whole cast rather than just one agent, means that choice was never on the table to begin with.
Nothing reaches the page without a second opinion
The third post in this series described how a citation gets checked before it reaches the page: was this actually retrieved, and does it actually support the claim. That checking is one instance of a pattern that runs across the whole workforce, not just the Explorer: in every pipeline that generates something, what’s written is checked by something other than itself before a person sees it, or before it becomes part of the record.
| Agent | Check | What it checks | What happens on failure |
|---|---|---|---|
| Explorer | Citation validator | Citation membership, whether the whole answer is grounded in what was retrieved, no diagnosis, recency | Retries with feedback, up to five times; a fixed, honest message replaces the answer if none pass |
| Explorer | Renderer | Whether each citation’s resource was actually retrieved | Drops the citation silently, rather than showing something unverified |
| Summariser | Grounding validator | Faithfulness to the source records, citation membership, no diagnosis | Retries up to three times, then the same fixed message |
| Document Filer | Confidence check | Whether the upload looks like a care record | Blocks the upload |
| Document Filer | Injection check | Whether the extracted text looks like an attempt to manipulate the agent | Blocks the upload |
| Document Filer | Description validator | Whether the generated summary is faithful to the uploaded document | Retries, then flags the summary as unverified rather than withholding it |
| Enquirer | Format and threshold checks | How strong the template match is | Blocks |
The shape repeats across every row: a generative step produces something, a separate step checks it against a narrower, more mechanical question, and the outcome of that check, not the generative step’s own confidence, decides what happens next. Some failures end in a retry loop with a hard stop and a fixed refusal. Others end in something being quietly dropped rather than the whole answer being blocked, or a document being rejected outright rather than filed with a caveat. Which behaviour fits which failure is a genuine design choice worth making deliberately, agent by agent, rather than applying one blanket response to every kind of failure.
flowchart LR
A("A specialist's synthesiser writes something") --> B("A separate validator checks it")
B -->|Fails, attempts remain| C("Retry with feedback")
C --> A
B -->|Fails, attempts exhausted| D("Fixed, honest message shown instead")
B -->|Passes| E("Reaches the page")
classDef base fill:#437c9f,stroke:#2f5f79,color:#ffffff,stroke-width:1px;
classDef highlight fill:#6dc067,stroke:#437c9f,color:#1f2d1f,stroke-width:1.2px;
class A,C,E base;
class B,D highlight;
linkStyle default stroke:#437c9f,stroke-width:1.2px;
Reading and writing don’t earn the same trust
Every specialist covered so far answers a question, which is a read. The Appointment Scheduler is the one member of the cast that writes, and it’s built asymmetrically because of it. One model call has the tools that search for and describe available slots; it can look, but it can’t book. A second, narrower call has exactly one tool available to it, the one that actually creates the appointment, and that tool is only ever reached after the person has confirmed a specific, locked choice presented as a chip, never from typed free text alone.
That split connects directly to the second post’s argument that asking is itself a form of access. If a question about a record is a read of that record, then a sentence that sounds like an instruction to book something is, on the same reasoning, dangerously close to being treated as the instruction itself. A locked confirmation closes that gap, turning “book the 3 o’clock slot” from something a model has to interpret into something a person has unambiguously chosen. It’s the one place in the whole ensemble where an agent is trusted to act rather than only to inform, and it’s built with a visibly firmer safeguard than any of the reading specialists carry.
Considerations worth weighing
A few further questions tend to arise in practice.
- How many specialists is the right number? Splitting capability across narrow agents makes each one easier to reason about, but every additional specialist is one more thing to route to, identify and audit. The right number is the one that matches the shape of the questions actually being asked, not a target to aim for.
- Does every agent need an identity? Granting one to every member of the cast, whether or not it ever touches the record, adds bookkeeping without adding accountability. Granting one only where something is actually read or written keeps the identity model matched to what’s really happening.
- How many attempts is enough? A retry loop with feedback gives a model a genuine chance to correct itself, but every retry costs latency, and a loop that never terminates isn’t a safeguard, it’s a stall.
- Does every failure deserve the same response? Dropping a single unverified citation, retrying a summary, and blocking a document outright are three different responses to a failed check, and it’s worth being clear about which failures call for which, agent by agent.
- Does a write need more than a read needs? A locked confirmation is a heavier safeguard than anything a read requires. Whether that asymmetry is justified depends on what’s actually being written.
- How exhaustive does a check need to be? A check that reads extracted text for an attempted manipulation catches an instruction expressed as ordinary text. It isn’t, on its own, a complete answer to every way an instruction might be hidden in a file, text laid beneath a rendered page in a colour that matches its background, for instance, only becomes checkable once image inspection joins text extraction as a further layer. The checks described in this post are illustrative of a pattern worth adopting, not a claim to have covered every way a document, or an answer, might be made to lie. Where the boundary of “checked” actually sits, and whether it’s wide enough for what’s genuinely at stake, is worth working through deliberately for your own design rather than assumed from the presence of a check at all.
Questions to ask of your own design
If you’re building a workflow of agents in front of clinical data, these may be worth asking:
- Is your chat interface really one agent, or a cast of specialists wearing one continuous conversation as a costume, and does your design reflect which it actually is?
- Does every agent in the workflow carry the identity that matches what it can actually do, no more and no less?
- Is the decision about what a question means kept separate from the decision about what’s allowed to happen next?
- Is a generated answer, or an uploaded document, checked by something other than the thing that generated or submitted it, before it’s treated as trustworthy?
- When a check fails, does the response match how serious the failure is?
- Does anything your system writes require a more deliberate confirmation than anything it merely reads?
- Have you been deliberate about where your own checks stop, or only about the fact that they exist?
The ensemble isn’t exempt either
Across this series, the argument has kept coming back to the same shape. An agent is a user of the application, so it needs the identity, the audit trail and the permissions any other user would have. A citation is a claim, so it needs to be checked, not merely displayed. A knowledge base built for a model still has to inherit the scoping and versioning built for everyone else. This post has simply pointed the same argument at the workflow as a whole: not one agent behaving well, but a cast of them, each scoped narrowly, each checked before it’s trusted, working together in a sequence that’s designed rather than assumed.
None of that makes the workflow special. It makes it exactly as ordinary, and exactly as accountable, as every individual part it’s made from.
The data used in the project to which this article relates was sourced exclusively from datasets that are known or assured to be synthetic, or have been intentionally created for illustrative purposes. The application described is a playground, built for experimentation, and has never been used in any real-life scenarios. It was created solely to demonstrate the concepts described in the article.
