"Send 150 to Bianca and show my balance."
That sentence is two jobs: move money and read an account. An early version of the assistant treated every message as exactly one intent, then forwarded it to a specialist model. The transfer path might run while the balance question was dropped. Each compound request became another branch in the routing table.
The diagram below shows the shape of that design. The router picked one specialist per message; the branches are mutually exclusive, so combining tasks required ad hoc exceptions.
Single-intent routing
Chat message
│
▼
Core API (Go)
│ gRPC
▼
AI service
│
▼
Intent classifier
│
▼
one chosen intent
│
┌────┼───────────────┬───────────────┐
▼ ▼ ▼ ▼
Transfer Wallet General Bill pay
The replacement runs an agent loop inside a durable workflow. The model calls a tool, reads the result, and decides what to do next. In the same turn it can resolve a payee name, prepare a transfer, and fetch a balance afterward.
A Go parent workflow hands each turn to a Python child workflow. Several tool calls can happen before the user sees a reply.
Agent loop
Chat message
│
▼
Core API (Go)
│
▼
Chat workflow
│ child workflow
▼
Assistant workflow (Temporal, Python)
│
▼
agent loop
think → act → observe
│
skills / tools / guardrails
│
▼
structured reply + actions
Load tools when they are needed
Giving the model every tool at once made prompts large and harder to follow. Tools were grouped into skills. The agent starts with a small set and loads more instructions and tools only when a task needs them.
For the transfer-and-balance example, the steps stay inside one conversation turn. The balance lookup follows the transfer result.
User: "Send 150 to Bianca and show my balance"
1. Resolve what "Bianca" means
2. Load transfer-related skill
3. Validate and prepare the payment
4. Execute transfer workflow
5. Load account skill
6. Fetch fresh balance
7. Return one coherent reply
Product documentation follows the same rule. A knowledge-base tool searches it when needed. Account data comes from backend queries. A document search cannot tell you a user's current balance.
Requests were also staggered across model providers, taking the first valid response. That helps when one provider is slow, but adds routing and cost decisions to every incident review.
Later candidates start only if no earlier provider has returned a valid response; provider health can change the order.
Model race for one agent turn
T+0.0s ── self-hosted model starts
T+2.0s ── fast inference provider starts if nothing has won yet
T+4.5s ── cloud fallback starts as the reliable last resort
Winner = first valid response
Losers = cancelled
Check what actually happened
The core backend owns money movement. The model requests actions through tools.
Replies are validated against those tool calls. A payment confirmation needs evidence that the action happened. A direct balance request needs a fresh balance query. If the reply fails a check, the agent gets feedback and retries.
Validation can reject a call, correct its arguments, or request another attempt. Tool checks happen before execution; reply checks compare the text with the recorded results.
Guardrailed execution
user request
│
▼
LLM proposes text + tool calls
│
▼
guardrails inspect:
- was the right tool called?
- was fresh data fetched?
- did text claim an action that never happened?
│
├──────── bad but fixable ─────► retry with feedback
├──────── bad and autocorrectable ► inject / rewrite tool call
└──────── valid ───────────────► backend executes side effect
These checks catch specific failures; they do not make model output inherently reliable.
The new path rolled out behind a feature flag while the old router stayed available. Rollback was possible, but debugging now required the active flags, model route, tools, and workflow path. Temporal also brought work around retries, workflow history, and deterministic execution.
The router was easier to operate. It was replaced because compound requests had become a normal part of the product.