Ship Validated Actions, Not Models

Everyone is shipping agents. Almost no one can put them in production.

That gap is not a capability gap. The models work. The agents demo beautifully. What fails is the last inch — the moment an agent stops describing the world and does something to a customer. In a bank, a wealth firm or an insurer, that inch is where the examiner, the CRO and the customer are all standing.

So we changed what we ship. Not a model. A validated action: an action that has already passed a value check and a compliance check, and can show its work.

1. The action carries the risk and the value — not the model

Model governance has trained us to inspect the wrong object. We review data lineage, security posture, drift dashboards, fairness metrics — and we get a green light on all of it. Then someone asks the only question that matters:

“Was this action right, and can you prove it?”

A model that looks green on every dimension still cannot answer that. The score was never the deliverable. The hard hold on an account, the SAR narrative, the retention email to your largest client — those are the deliverables, and those are what carry both the risk and the return.

Once you accept that, the design question changes. It is no longer “how accurate is the model?” It is “what has to be true before this action is allowed to execute?”

2. One agent is not enough

Three patterns dominate the market, and each one fails the last inch in its own way.

• Ungrounded agents hallucinate. LLM-only agents guess. In finance, a wrong-but-confident, inconsistent answer is unshippable — and confidence is exactly what makes it dangerous.
• Automation tools skip the decision. Process-automation agents automate the doing, not the judgment the doing depends on. They execute faster; they do not execute more wisely.
• No audit trail means no deployment. When compliance, drift and human oversight are afterthoughts, the pilot never crosses into production. Not because it failed — because nobody could sign it.

Adding a second LLM that reviews the first one does not fix this. Two similar agents make correlated mistakes. What you need are two checks that are independent because they are asking genuinely different questions.

3. Two independent checks, on every action

Before an action reaches a queue, it goes through two agents that answer to different masters.

• Focus-Group agents — Will the customer actually respond? If the check fails: The action is dropped — it would have burned an advisor hour or annoyed a good customer.
• Expert-Panel agents — Is it compliant and proportionate? If the check fails: The action is returned with a reason, and the reason is on the record.

The Focus-Group agent is a distilled focus population — personas built from your own segments, not generic archetypes — and it answers the value question. In retention it is “will this client respond?”. In fraud it is sharper: “will a legitimate customer be harmed by this?”

The Expert-Panel agent is your organisation: the SIU investigator, the AML officer, the compliance lead, the head of advisory, each with the mandate and the veto they would have in a real committee, and each measured against stated goals and KPIs. It answers the compliance question and signs off.

Only actions that pass both execute. The rest are dropped or returned with a reason — and the reason is logged, not discarded.

4. What has to be true underneath

A validation layer is only as good as what it validates. Ours sits on four non-negotiables:

• Explainable ML with a reason code on every score, produced by a companion explanation model that names patterns in business language — not a global feature ranking.
• Humans in the loop throughout. Risk officers own the label definition, can approve and edit the actions, prompts and data, and can gate any step. The platform automates the documentation and the path to production — not the judgment.
• Auditability by default. Every validation is a logged meeting you can reopen months later and read back question by question. Model cards, data dictionaries and compliance documents are generated, not written after the fact.
• Your environment, your keys. Cloud, on-premise or hybrid; bring your own keys and your own LLMs. The platform is LLM-agnostic, so no vendor decision is locked into your control framework.

5. The same predictions, with validation on top

The cleanest way to see what the layer is worth is to hold the model constant and switch validation on. Identical predictions, corrected action layer:

• 94% → 42% needless manager alerts in wealth retention, with 27% of the book cleared as “no action” and corrupted-data cases caught with zero false alerts.
• 20 → 0 false SAR exposures on legitimate customers in the top account-takeover tier — same detector, corrected action layer, 100% auditable rule trail.
• 98 → 4 top-tier AML SAR escalations: 96% fewer, every one corroborated, and “source of funds” tipping-off phrasing removed from 99% of narratives down to 0.4%.

The underlying agents pay for themselves on their own. Account takeover: +37% investigation efficiency on test and 53% on hold-out, worth USD 480K and USD 2.5M more fraud caught respectively, with true positives moving from 50% to roughly 80% and false-negative cost falling from USD 1M to USD 235K. Wealth retention: 15% attrition reduction, a 20% save rate, USD 8M NPV and up to 32× ROI in the first year. Voice of customer: 100% severity accuracy, a month of complaints classified in one minute, and 42 Level-3 cases recovered from Level 1.

6. This is what turns a pilot into production

A scoped proof of value runs on one lever, in your environment, in roughly 72 POV hours: prepare the data, label and business KPI; build the grounded model and its explanation layer; validate the actions with the two agents; A/B test against the incumbent process; go live with the compliance documentation already written.

Start small, together. One use case at a time, data cleaned and improved as you go, each win funding the next. That is not a modest ambition — it is the only sequence that survives a control environment.

One question. Which AI action in your shop executes today without a value check and a compliance check in front of it?