- What the test actually asked
- The money table: what a passed task costs
- Zero, exactly where it matters most
- The board moved this week, and the math barely did
- What a Canadian buyer does with a 12% world
For a test that works on real files instead of a benchmark, see One energy firm tested three AI tools blind. The method matters more than the winner.
Start with the only number on the invoice that matters. On November 2, 2026, a research firm handed the world’s best AI models 206 real professional tasks. The best score any model posted was 12%. Not 12% below the demo you saw at a conference. A flat 12% of the work, finished, full stop.
The direct answer: the test is GDPx, built by SemiAnalysis with 14 specialist data partners. It gives models up to eight hours inside realistic work environments across 11 professional fields. At launch, Claude Opus 5.5 topped the board at 12.0%, and every model scored zero under strict grading in finance and legal. A newer entry has since pushed the top score to 16.4%. Either way, the buyer funds the supervision.
What the test actually asked
Most benchmarks ask questions. This one assigns jobs, inside simulated workplaces built by firms that sell training environments to the labs. The 11 fields are chip design, hardware design, cybersecurity, biology, finance, legal work, quantitative trading, recursive self-improvement, materials science, alignment and software engineering. Thirteen models were published on the first leaderboard, each scored on its first attempt, averaged over three runs. A plausible draft that fails the acceptance test scores the same as a blank page.
The finance set shows the standard the tasks were held to. One environment rebuilt a real $880 million private-equity acquisition from roughly 4,000 anonymized files. The model had to find what mattered, do the analysis and produce work a deal team could sign. No model passed the public quality-of-earnings task in that set. These are not trick questions built to embarrass the machines. They are the expensive hours your firm bills for, or pays for.
The money table: what a passed task costs
Scores are the headline, but the cost column is the story for anyone signing purchase orders. Opus 5.5 averaged US$51.41 per attempted task, about CAD 73 at the Bank of Canada’s November 9 rate. Divide the attempt price by the 12.0% pass rate and a finished task costs roughly US$428, about CAD 605. Google’s Gemini 4 Carbon scored 10.9% at US$16.29 per attempt, about CAD 23, which works out near CAD 211 per finished task. The cheapest finisher on the board is the model most buyers cannot yet buy for general use.
GDPx: what the best models scored, by field
| Item | Figure |
|---|---|
| Best overall score (Opus 5.5) | 12% |
| Tasks in the test | 206 |
| Fields covered | 11 |
| Finance score | 0% |
| Legal score | 0% |
| Current leaderboard top (Fable 5.2) | 16.4% |
That arithmetic should be stapled to every agent proposal on your desk this quarter. Vendors quote the seat price or the per-task price, never the per-finished-task price. Your own results will differ from a benchmark built on frontier work, in both directions. The discipline is to run the division on your numbers before the renewal, not after it. Beacon’s 9.6 AI hours only count after you subtract review time makes the matching point on the labour side: gross hours are marketing, net hours are money.
Zero, exactly where it matters most
The most valuable sentence in the whole release is the grading footnote. In finance and legal, strict pass-or-fail grading returned zero for all 13 models, every one. The publishers switched those two fields to partial credit because a column of zeros carried no signal. Sit with that if your firm sells advice, moves money or signs opinions for a living. The two fields where an error becomes a liability are the two fields where the machines finished nothing.
Canada has no shortage of firms about to learn this line item by line item. RECO AI guidance reaches the AI already inside your brokerage CRM shows regulated advice work absorbing AI under a regulator’s eye. The regulator’s question and the benchmark’s question are the same question. Who checks the work, and who answers for it when the work is wrong? On the current evidence, the answer is your staff, on your payroll, under your licence.
The board moved this week, and the math barely did
One update belongs in this piece before any reader quotes the headline at a board table. The live leaderboard now carries a newer Anthropic entry, Claude Fable 5.2, at 16.4% and US$88.15 per attempt. That is real progress in eight days, and it changes the buyer’s math less than the headline suggests. A 16.4% finisher still leaves five of six tasks unfinished, at roughly CAD 759 per completed one. The publishers also note that safety classifiers held back two leading models in three fields. The honest range for the best machine on professional work today is somewhere between 12% and the low twenties.
Evaluation is becoming a buying skill in its own right, separate from the tools themselves. OpenAI LifeVal grades AI agents on errands that stop short of the CRA login grades agents on everyday work rather than frontier work. Read together, the two tests bracket the market you are actually shopping in. Errands, mostly handled. Professional judgment, mostly not, at any price yet quoted.
What a Canadian buyer does with a 12% world
None of this argues for waiting on the sidelines with clean hands. BDC Mission AI will vet Canadian AI tools while BDC backs 300 AI firms shows the country’s own capital moving toward vetted adoption, not abstinence. The firms that win this cycle will buy drafting speed and fund checking capacity in the same budget line. They will treat the model as a fast junior analyst who never sleeps and never signs. The signature, and everything the signature carries, stays with a person whose name is on the door.
So here is the dated action, and it fits before any January renewal conversation. Take 20 real tasks from your own floor, finished ones with known right answers, and run your current stack against them. Score the passes the hard way, finished or not, and divide your true cost by the passes only. If your number lands near the benchmark’s, your 2027 budget needs a supervision line before it needs more seats. The best AI in the world scored 12% on real professional work. Budget like you believe it.
