Kimi K3 wins its own benchmark table, so test it on your own 20 tasks

Kimi K3 lists at $3 and $15 per million tokens and tops some of its own tests. Priya Chen shows how to run a cheap 20 task trial on your real work.
A colour tape measure with centimetre markings laid across a wooden surface.

About $9. That’s what we’d spend testing Kimi K3 on 20 of your own tasks before anyone in your company says the word “migrate”. Moonshot AI’s launch table puts K3 near the top labs on the lab’s own tests, and a seller’s tape measure is a fine reason to be curious. It’s a poor reason to switch models this week.

Key points

  • Kimi K3 lists at $3.00 per million input tokens on a cache miss, $0.30 on a hit and $15.00 per million output. Mid-tier, and Moonshot itself says K3 “still trails” the leaders overall.
  • The same benchmark gives different numbers on different pages, even from the same lab.
  • A blind trial of 20 real tasks, three runs each, settles more than any table.
  • Write your switching rule down before you look at results.

The price, read against the shelf

K3 comes from Moonshot AI, a Chinese lab. It’s a mixture-of-experts model with 2.8 trillion total parameters and a 1 million token context window, with weights promised by July 27. Moonshot’s launch post concedes it trails Claude Fable 5 and GPT-5.6 Sol overall.

The rate card matches what Anthropic lists as the standard price for Claude Sonnet 5 from September 1, and Anthropic’s introductory Sonnet 5 rate on July 17 was $2 and $10. So K3 isn’t the cheapest thing on the shelf, and our look at Opus 4.8’s unchanged price shows why list prices mislead anyway.

Advertisement

Independent benchmarker Artificial Analysis scores K3 at 57 on its Intelligence Index, behind Fable 5 and GPT-5.6 Sol, at about $0.94 per index task against about $1.04 for Sol. A modest discount. The cache-hit price is a tenth of the miss rate, so a workflow that sends the same long instructions every time could run much cheaper; the launch post doesn’t say which tasks qualify. A saving that size disappears the first week your staff spend extra time fixing answers.

Why one benchmark gives three answers

The test, its version, the scoring setup and the reasoning setting all move a score, and each lab reports the combination that flatters it. Moonshot’s launch post gives K3 a BrowseComp score of 90.4. The model card, updated later in July, lists 91.2 for K3 and 90.4 for Sol. Same lab, two pages.

Across labs it gets wider. Moonshot’s table puts Sol at 73.0 and Fable 5 at 70.0 on DeepSWE. OpenAI’s own scorecard, published July 17, has 72.7% and 69.9% on DeepSWE v1.1. And on July 8 OpenAI said about 30% of the tasks in the SWE-Bench Pro coding benchmark were broken, withdrawing its earlier endorsement. With a yardstick that stretchy, a one-point lead means nothing. We read K3’s table the way we read the GLM-5.2 cybersecurity benchmark claims in June: an invitation to test.

Our $9 trial, step by step

  1. Pick 20 real tasks from last month. Ask the support lead for the five worst emails and the bookkeeper for the five worst documents. The scanned invoice belongs in there. Easy work makes every model look good.
  2. Run each task three times on K3 and three times on your current model. One lucky answer proves nothing.
  3. Have someone score the outputs blind. Count corrections, time them, and note refusals and invented figures.
  4. Prefer honesty to confidence. Right 16 times and unsure on the rest beats right 17 times and confidently wrong on three.

The cost, as our own illustration: say each run sends 30,000 tokens in and gets 4,000 back. At K3’s list rates that’s about $0.09 for input and $0.06 for output, $0.15 a run. Sixty runs, $9.00, less with cache hits. Scoring time will cost far more than the model, which is why the hidden time tax of AI matters, and our breakdown of AI costs for Canadian business shows where the rest of the bill hides.

Decide the switching rule first

Ours: K3 has to match your current model on usable answers out of 20 and beat it on cost per usable answer by enough to pay for the move, meaning the switching work, the prompt rewrites and your staff’s habits.

A made-up scorecard shows the shape. Your current model gives 17 usable answers and K3 gives 15, with tokens a third cheaper. Stay put: two extra bad answers in 20 cost more in review time than the roughly $3 saved. Flip the scores and the conversation changes even at the same price. The rule ignores leaderboards and open weights. It cares about the 20 pieces of work you do.

The sceptic says open-weight releases get tuned to look good and a working model deserves protecting. Tables do get tuned. Nobody can tune for your 20 tasks in advance, though, so the trial answers the sceptic better than an argument. And if your current model already handles everything at a price you like, skip the trial and write down why.

Before any move, check where a hosted API processes customer data, and read the Kimi K3 License on the model card before you think about self-hosting. At 2.8 trillion parameters, that’s a data-centre job. Keep a tested fallback model ready, as Anthropic’s June Fable withdrawal showed.

We haven’t run K3. Every score here comes from Moonshot, OpenAI or Artificial Analysis, and prices are list prices on July 17. Keep your scores, and the next shiny table takes an afternoon to answer. Write the 20 tasks down this week.

Frequently asked questions

How much does Kimi K3 cost?

Moonshot lists $3.00 per million input tokens on a cache miss, $0.30 on a cache hit and $15.00 per million output tokens. Check the current price before budgeting.

How do I test an AI model for my business?

Use 20 real tasks, run each three times, and score the outputs blind. Count the corrections and time them, then compare cost per usable answer.

Archive entry dated July 17, 2026, written and fact-checked on October 8, 2026. Sources are linked on the claims they support.

Total
0
Shares
Prev
Startupfest 2027 founders who show their AI gross margin will stand out on stage
Montreal waterfront buildings along the river, setting of Startupfest 2027

Startupfest 2027 founders who show their AI gross margin will stand out on stage

Startupfest 2027 runs July 7 to 9 in Montreal

Next
OpenAI’s CFO wants you to count the cost of each finished task, and you should, with your own numbers
A calculator and a pen resting on a sheet of paper.

OpenAI’s CFO wants you to count the cost of each finished task, and you should, with your own numbers

Cost per successful task is a better AI budget metric than price per token, but

You May Also Like