tl;dr
We ran seven frontier models through the same 500 compliance cases. This experiment proved that the model itself is not the most consequential part of the build-versus-buy decision. Bretton’s agent harness dramatically shifted how the agents worked: agents reached the correct answer more often, answered more consistently, and provided answers of far higher quality, especially on the most complex cases.
Customer outcomes with the Bretton harness
Estimated model cost reduction
28% cheaperacross 500 investigations.
Models cost less inside the Bretton harness than on their own, while also fostering a higher consistency rate.
Estimated analyst hours saved
72 hoursper 100 investigations.
Based on a four-hour manual investigation and compared with provider harness performance.
Results
OVERALL
7 / 7
Models improved performance.
Each model passed more required checks inside the Bretton harness than in native API format or inside its own lab’s agent.
ACCURACY
95%
of required checks were passed.
Inside Bretton's harness, 95 in 100 required checks were complete and correct, against 87 in 100 in native API format.
CONSISTENCY
97%
of cases were correctly escalated.
Inside Bretton’s harness, models escalated 97% of cases needing review, versus 78% without, indicating improved consistency.
CONTEXT HANDLING
98%
of tasks with the largest context passed.
On files with 13 to 29 documents. The provider harness passed 86/100. Models in native API format passed 81/100.
Purpose
We explored the question every bank asks
“Why not use chatGPT, Gemini, or Claude to complete my back-office tasks?”
Every bank that evaluates an AI vendor asks it, and it is a prudent question. The frontier models are highly capable, inexpensive, and a competent engineer can write the prompts in about a week. Some teams go one step further and use OpenAI Codex or Claude Code, both of which are wrapped in its lab’s own agent harness. This is how the code-specific AI applications are able to conduct tool use and multi-step execution.
In August, we argued that building the agent was never the hard part. The most difficult part is building a system an institution can rely on at production volume, inside its own controls, for many years without further internal engineering work. That was our argument from experience. This benchmark is the unbiased evidence to prove that argument.
Our findings are clear. A model running in native API format, or a general-purpose coding agent, is consistently worse across every metric as compared to the model governed by Bretton's agent harness.
We measured this difference in three ways:
- Accuracy: is the output correct?
- Consistency: do different models reach the same answer for the same case?
- Context handling: does quality remain high even as the case grows more complicated?
These are the three properties a bank would otherwise have to engineer, test, and maintain.
Methodology
- Seven frontier models from three labs: Claude Opus 5, Claude Sonnet 5, GPT-5.6 Sol, GPT-5.6 Luna, GPT-5.6 Terra, Gemini 3.7 Flash and Gemini 3.1 Pro.
- 500 cases, built to look like a real onboarding queue. 250 companies (KYB) and 250 people (KYC). Each case contains one onboarding investigation. A task is a required piece of work within it, such as sanctions screening or adverse-media review.
- 250 real subjects, grounded in public records. Companies, executives, politically exposed people and sanctioned individuals across the UK, US, Colombia, Denmark, Saudi Arabia, Armenia and more. Every fact in the answer key is tied to a primary source.
- 250 fictional subjects with document packs. 2,000+ synthetic documents across 29 types, including registry extracts, ownership charts, bank statements and identity documents. These contain known defects: an expired passport, a bank account under a trading name, or inconsistent ownership records. This lets us test whether an investigation catches specific issues.
- Three conditions:
- Native API format: one prompt, one context window. The model received every compliance task, the full document pack, and the tools in a single session, and wrote the report directly in response.
- Provider harness: the same session inside the lab's own general-purpose agent, Claude Code for the Claude models and OpenAI Codex for the GPT models. It adds real tool use and multi-step execution, but no compliance-specific control. Google has no equivalent, so the two Gemini models ran in the other two conditions only.
- Bretton agent harness: the same inputs inside Bretton's harness. It splits the work into discrete compliance tasks, runs each in its own context, and will not close the file until every task has a result on the record.
Three ways to run the same case.
Same model, same checks, same documents, same tools.
Native API format
One prompt
The case is evaluated in one shot.
The model writes the report.
Provider harness
A general-purpose agent
Research · act · repeat
The agent chooses the next step.
The agent uses tools and decides what to do next.
Bretton harness
Compliance controls built in.
Related checks share context.
Required work and unresolved gaps stay on the record.
We evaluated the results using a pre-determined answer key. For every case we wrote down what a correct report has to say: the risks it must flag, the checks it must record as clean, and the claims it must not make.
Accuracy
Bretton's harness scores more accurately on every task
Inside Bretton's harness, 95 in 100 required checks were complete and correct, against 87 in 100 in native API format.
Every model passed more graded items inside Bretton's harness, by 6.5 to 15.3 percentage points. The four models with the lowest scores in native API format gained the most, 11 to 15 percentage points each.
What this means for your team:
Fewer compliance tasks come back needing a fix. About 1 check in 8 came back to a reviewer for rework using models in native API format, whereas with the Bretton harness, only 1 in 20 required rework. That cuts the review queue your analysts must work through every morning by almost two thirds.
How we judge accuracy.
The report must match our answer key: it flags known risks, records checks that came back clean, and only makes claims the evidence supports.
Every model improved with the Bretton harness
Required checks passed · percentage · scale 75–100%
Even by task, it's clear that the Bretton harness improves accuracy
Compared with models running in native API format, pass rates by task rose approximately 38% on sanctions, 22% on adverse media, 18% on political exposure, and 13% on geographic risk. Domain history rose 31%, and terms and conditions 18%.
Accuracy by task
Required checks passed · percentage · scale 70–100%
And the provider harness?
Five models ran in all three conditions. Their pass rate was 87% in native API format, 87% inside the provider harness and 95% inside the Bretton harness. Claude Code and Codex moved the result by 0.2 percentage points before rounding. The Bretton harness moved it by approximately 8 percentage points.
How Bretton’s harness succeeds:
In native API format, most failures were due to omissions, not wrong answers. Of every 100 graded items, the models left out 15.3 on average: examples include a required finding not stated, a clean result not recorded, or a run that produced no report. Inside the harness, omissions fell to an average of 2.6/100. The harness runs each compliance task as its own step, with its own context and sources, and it does not close the file until each one has a stated result.
Consistency
Bretton’s harness reduces variability
Inside Bretton’s harness, models escalated 97% of cases needing review, versus 78% without, indicating improved consistency.
Two analysts reviewing the same file should reach the same conclusion. When they do not, someone has to trace the discrepancy and decide what needs action. An AI investigation faces the same challenge, with one more variable: the underlying model can change. Your team should not have to reconcile a different set of findings every time it does.
To test that, we gave seven models the same KYC and KYB cases, first running in native API format and then inside Bretton’s harness. We compared their verdicts on required escalations and graded findings across models, not repeat runs of one model. Inside the harness, the models agreed more often, leaving fewer conflicting findings for your reviewers to sort through when the model changes.
- Cases requiring review are escalated more reliably. Inside Bretton’s harness, models escalated 97% of cases requiring review, versus 78% running in native API format. All seven models agreed on those required escalations 48% more often.
- Planted defects surface more reliably. Inside Bretton’s harness, models raised planted defects 15% more often than without Bretton’s Harness.
- Fewer conflicting findings to reconcile. Across all graded findings, all seven models reached the same verdict 14% more often inside Bretton’s harness than running in native API format.
What this means for your team:
Your reviewers can rely on more consistent findings, whichever model runs the case. Cases requiring review are more likely to be escalated to them, and a change in model is less likely to produce a conflicting conclusion.
Cases that required a review were correctly escalated.
Bretton's harness ensures that reviews are less likely to be missed.
The frequency of case escalation agreement across all seven models.
Bretton's Harness ensures that escalation decisions are more consistent.
Defects planted were correctly detected.
Bretton's harness ensures that case defects are less likely to be missed.
The frequency of case verdict agreement across all seven models.
Bretton's Harness ensures that case verdicts are more consistent.
How Bretton’s harness succeeds:
In native API format, seven in ten model disagreements arose because some models left a finding blank while others answered it. Bretton runs each check as a required step, with the same instructions and output structure whichever model runs. Inside Bretton’s harness, all seven models reached the same verdict on graded findings 14% more often than when running in native API format.
Context handling
Bretton's harness is built for complex cases
On files with 13 to 29 documents. The provider harness passed 86/100. Models in native API format passed 81/100.
A complex KYC or KYB case may include registry records, IDs, bank statements, ownership documents, and source-of-wealth evidence. Even as the amount of data grows, the agent still has to find the right evidence for each check and account for every required party. Context handling is whether that work stays complete and accurate across all of this complexity, not just whether the model can accept a long prompt.
It matters most on the files that take analysts longest: a large document pack, a long ownership chain, and a list of directors and owners who each need screening. This section compares the same five models, again running all three ways: in native API format, inside the provider harness, and inside the Bretton harness.
- Every owner and director gets screened. Inside the Bretton harness, 98% of reports screened every related party, and this high rate was maintained even as the list grew. In native API format or inside the provider harness, only one report in three had everyone screened, and for companies with nine or more parties, most of the roster went completely unscreened.
- Quality holds on the largest files. On packs of 13 to 29 documents, models in native API format passed 81% of their checks. The provider harness: 86%. The Bretton harness: 98%. The task that yielded the largest gap was sanctions. In native API format and inside the provider harness, the check passed on about half of runs. Inside the Bretton harness, 98.9%.
- Required work is completed. On large files, Bretton carried investigations through completion. In native API format, the model did not produce a report for one in eleven runs.
What this means for your team:
The cases with the most context are the ones that carry the most risk, and the ones your analysts spend the most time on. Bretton handles them to the same standard as the simplest cases, so you can trust the agent with the toughest investigations, not just the easy ones.
Related parties screened
Coverage as the number of related parties increases · scale 0–100%
Checks passed on 13–29 document packs
Case study
To showcase the difference between how an agent tackles a case, we selected one of the most complex investigations that we ran for this benchmark. Bhatt Realty Private Limited, a fictional Indian property company, came with 26 documents and 81 pages: registry records, articles, an ownership chart, annual accounts, three bank statements, proof of address and a source-of-wealth letter. We planted a major problem that can only be discovered across documents: the registry lists the company as dissolved, while its bank statements show it still trading.
Bhatt Realty Private Limited
9 graded compliance tasks · GPT-5.6 Sol · 26 documents · 81 pages
NATIVE API FORMAT
Did not finish any of the tasks
The run stopped before it produced a report. Nothing ended up reaching the investigation file.
PROVIDER HARNESS · OPENAI CODEX
Finished, but left four tasks blank
It caught the dissolved-status conflict. But sanctions and political exposure were "not established", and it recommended screening "the entity and all principals for sanctions, PEP and adverse media" instead of running it. India’s high-risk rating and the industry line were never stated.
INSIDE THE BRETTON HARNESS
Ran every task to the end
“No sanctions or watchlist match was found for Bhatt Realty Private Limited or any document-derived roster member.” It flagged the dissolved status, rated India high risk and screened the owner.
How Bretton’s harness succeeds:
Bretton agents begin by tackling the identity of the customer or business in question: extraction builds the roster of related parties before any research, and each party gets an investigator that owns its screening. The evidence lives outside the prompt: documents become searchable files that each investigator retrieves when it needs them, so a 29-document pack gets the same level of attention as a 9-document one. Finally, prompting is not the enforcement lever: every check must resolve, or stay visibly open, for the file to be considered closed.
Implications
Building the agent was never the hard part. A model can write a report. A provider harness gives it tools and lets the agent decide what to do next. However, we found that provider harnesses did little to actually improve accuracy, consistency or quality on large files. Bretton’s agent harness addressed and improved all three. To summarize, these are the features that make it possible:
- A harness that enforces required work. Each check runs as its own step, and it must resolve or stay visibly open before the file closes. This is the checklist and attention-anchoring layer we described in our piece on Bretton’s agent harness.
- The documents live outside the model's memory. Instead of loading the entire case pack into one prompt, the harness stores it as searchable files, and each check retrieves only what it needs. That is why quality holds on the largest files.
- Clean results on the record, so a reviewer can tell “clean” from “not checked”.
- Visible failures. A crash or a timeout must surface as a gap.
- A repeatable test of quality: The grader in this study compares reports against a fixed answer key, checking for missed risks, unrecorded clean results and unsupported claims. Building means maintaining the test cases, answer key and grading rules as the agent evolves.
Buy means these controls arrive built in, with Bretton responsible for ongoing testing and maintenance. Build means your team owns that work for as long as the agent runs. If you would like to learn more, we are always open to having a conversation!