A colour tape measure with centimetre markings laid across a wooden surface.

Kimi K3 wins its own benchmark table, so test it on your own 20 tasks

Kimi K3 lists at $3 and $15 per million tokens and tops some of its own tests. Priya Chen shows how to run a cheap 20 task trial on your real work.

Kimi K3 is worth being curious about and a poor reason to switch models this week. Moonshot AI’s launch table puts it near the top labs on the lab’s own tests, and that table is the seller’s tape measure. Spend about $9 on 20 of your own tasks before anyone in your company says the word “migrate”.

That’s the position. The rest is why the table can’t settle it and how a cheap trial does.

Key points

  • What does Kimi K3 cost, and is that cheap? Moonshot lists Kimi K3 at $3.00 per million input tokens on a cache miss, $0.30 on a cache hit and $15.00 per million output tokens, with a 1 million token context window.
  • Why do the same benchmarks give different numbers? Because the test, the version, the scoring setup and the reasoning setting all move the result, and each lab reports the combination that flatters it.
  • How do you test Kimi K3 for about $9? Pick 20 real tasks from last month’s work, run each three times on K3 and three times on the model you use now, and have someone score the outputs without knowing which model wrote them.
  • What result should make you switch? Decide before you run the trial, or you’ll talk yourself into whatever the numbers say.

What does Kimi K3 cost, and is that cheap?

Moonshot lists Kimi K3 at $3.00 per million input tokens on a cache miss, $0.30 on a cache hit and $15.00 per million output tokens, with a 1 million token context window. That’s a mid-tier price, not a bargain. Moonshot’s own launch post concedes K3 “still trails” Claude Fable 5 and GPT-5.6 Sol overall.

Advertisement

K3 comes from Moonshot AI, a Chinese lab. It’s a mixture-of-experts model with 2.8 trillion total parameters, and the weights are promised by July 27. The rate card matches what Anthropic lists as the standard price for Claude Sonnet 5 from September 1. Anthropic’s introductory Sonnet 5 rate was $2 and $10 on July 17, so K3 isn’t the cheapest thing on the shelf, and our look at Opus 4.8’s unchanged price shows why a list price alone misleads.

Independent benchmarker Artificial Analysis scores K3 at 57 on its Intelligence Index, behind Fable 5 and GPT-5.6 Sol, at about $0.94 per index task against about $1.04 for Sol. A modest discount. One wrinkle is the cache-hit price of $0.30, a tenth of the miss rate, so a workflow that sends the same long instructions every time could cost much less than the headline suggests. The launch post doesn’t say which of your tasks would qualify. Either way, we wouldn’t rebuild a workflow for it, because a saving that size vanishes the first week your staff spend extra time fixing answers.

Why do the same benchmarks give different numbers?

Because the test, the version, the scoring setup and the reasoning setting all move the result, and each lab reports the combination that flatters it. Moonshot’s launch post gives K3 a BrowseComp score of 90.4. The model card, updated later in July, lists 91.2 for K3 and 90.4 for Sol. Same lab, two pages, two stories.

Across labs the gap widens. Moonshot’s table puts Sol at 73.0 and Fable 5 at 70.0 on DeepSWE. OpenAI’s own scorecard, published on July 17, has them at 72.7% and 69.9% on DeepSWE v1.1. Close, not identical.

The test itself can be bad. On July 8 OpenAI said about 30% of the tasks in the SWE-Bench Pro coding benchmark were broken, and it withdrew its earlier endorsement. When the yardstick stretches like that, a one-point lead means nothing. We read the K3 table the way we read the GLM-5.2 cybersecurity benchmark claims in June, as an invitation to test and not a verdict.

How do you test Kimi K3 for about $9?

Pick 20 real tasks from last month’s work, run each three times on K3 and three times on the model you use now, and have someone score the outputs without knowing which model wrote them. Three runs matter because one lucky answer proves nothing.

Count corrections and time them. Note, too, whether a model refused a task it should have done or invented a figure. A model that’s right 17 times out of 20 and confidently wrong on the other three is worse than one that’s right 16 times and says it isn’t sure on the rest.

Advertisement

Put the ugly tasks in. The scanned invoice, the angry customer email. Easy work makes every model look good, and a trial made only of easy work will tell you what the launch table already did. Ask whoever handles the support queue for the five worst emails of the month, and whoever does the books for the five worst documents. Those are the tasks where models actually part ways, and where a wrong answer reaches a customer.

Here’s the arithmetic, and it’s our illustration, not a measurement. Say each run sends 30,000 tokens in and gets 4,000 back. At K3’s list rates that’s about $0.09 for input and $0.06 for output, so $0.15 a run. Sixty runs come to $9.00, less with cache hits. Your time scoring the answers will cost far more than the model, and that’s the number to guard. Our piece on the hidden time tax of AI explains why, and our breakdown of AI costs for Canadian business shows where the rest of the bill hides.

What result should make you switch?

Decide before you run the trial, or you’ll talk yourself into whatever the numbers say. Our rule is simple. K3 has to match your current model on usable answers out of 20, and it has to beat it on cost per usable answer by enough to pay for the move itself.

That means the switching work, plus the prompt rewrites and the habits your staff have built around the old model.

Here’s a made-up scorecard to show the shape of it, not a result. Your current model gives 17 usable answers out of 20 and K3 gives 15. K3’s tokens cost a third less. You’d still stay put, because two extra bad answers in 20 cost more in review time than the roughly $3 that a third off the token bill saves. Flip the scores, 17 for K3 and 15 for the incumbent, and the conversation changes even at the same price.

Notice what the rule ignores. It doesn’t care who topped which table, or whether the weights are open. It cares about the 20 pieces of work you actually do.

Is a trial worth the staff time if your current model works?

Often it isn’t, and that is a legitimate result. Open-weight releases do get tuned to look good on benchmark tables, but nobody can tune for your 20 tasks in advance. A short trial answers the question better than an argument, and the token bill usually costs less than a team lunch.

Advertisement

Often it isn’t, and that’s a legitimate result. The sceptic’s case is that open-weight releases get tuned to look good and a working model deserves protecting. Half of that is right. Tables do get tuned. But nobody can tune for your 20 tasks in advance, so the trial answers the sceptic better than an argument can, and the token bill costs less than a team lunch.

If your current model already handles every task you care about at a price you’re happy with, skip the trial and write down why. Revisit when Moonshot’s pricing or independent scores on messy business work change.

Two cautions before any switch. If your tasks include customer data, check where a hosted API processes it before you send anything. And read the Kimi K3 License on the model card before you think about self-hosting; a 2.8 trillion parameter model is a data-centre job, not a desktop install. If you do move, keep a tested fallback model ready, as Anthropic’s June Fable withdrawal showed.

We haven’t run K3. Every score here comes from Moonshot, OpenAI or Artificial Analysis, and they disagree in small ways. Moonshot’s launch post doesn’t print the full benchmark table, so we can’t say exactly where K3 loses. Prices are the vendor’s list prices on July 17 and may change.

Keep the scores. Next time a new model arrives with a better table, you’ll rerun the same 20 tasks in an afternoon, and the answer will be yours instead of the vendor’s.

Write the 20 tasks down this week, before the weights land.

Frequently asked questions

How much does Kimi K3 cost?

Moonshot lists $3.00 per million input tokens on a cache miss, $0.30 on a cache hit and $15.00 per million output tokens. Check the current price before budgeting.

Is Kimi K3 better than Claude Fable 5?

Moonshot says it beats Fable 5 on some tests and trails it overall. That’s the vendor’s claim, so test it on your own tasks before deciding.

How do I test an AI model for my business?

Use 20 real tasks, run each three times, and score the outputs blind. Count the corrections and time them, then compare cost per usable answer.

Archive entry dated July 17, 2026, written and fact-checked on October 8, 2026. Sources are linked on the claims they support.

Total
0
Shares
Prev
AI Adoption in Canada Tripled in Two Years to One in Five
Small business owner photographing a sneaker for an online listing beside her laptop and packed orders.

AI Adoption in Canada Tripled in Two Years to One in Five

AI adoption in Canada hit 19

Next
OpenAI’s CFO wants you to count the cost of each finished task, and you should, with your own numbers
A calculator and a pen resting on a sheet of paper.

OpenAI’s CFO wants you to count the cost of each finished task, and you should, with your own numbers

Cost per successful task is a better AI budget metric than price per token, but

You May Also Like