Somebody on your team will ask whether a free model can find security bugs as well as the paid ones. On the GLM-5.2 cybersecurity numbers, the answer is maybe, on one bug type, and nobody has checked it on your code. Semgrep’s June 22 benchmark scored GLM 5.2 at 39% F1 on insecure direct object reference flaws, against 37% and 28% for two Claude Code setups. Run a 20-bug test before you buy or cancel anything.
The headline said a free Chinese model matched Mythos. The table has no Mythos row.
Key points
- What did the GLM-5.2 cybersecurity tests measure? One bug class on one dataset. Semgrep’s benchmark scored detection of insecure direct object reference flaws, where one user can reach another’s data by changing an ID, using real open-source apps.
- Where did the claim of matching Mythos come from? From press coverage and from the lab, not from the tests.
- Does the wrapper matter more than the model? On this evidence, yes. The top row of Semgrep’s table is its own pipeline running GPT 5.5 at 61%, with Opus 4.8 in the same pipeline at 53%.
- What does a fair bake-off on your own code look like? It takes about a day and gives you the number that matters, your cost per true finding.
What did the GLM-5.2 cybersecurity tests measure?
One bug class on one dataset. Semgrep’s benchmark scored detection of insecure direct object reference flaws, where one user can reach another’s data by changing an ID, using real open-source apps. F1 balances catching real bugs against false alarms. GLM 5.2, given a prompt and some hints, scored 39%. Two Claude Code setups scored 37% and 28%.
Semgrep’s authors say plainly that this is “one task, one dataset, one run.” The open-weight models also got a search strategy and pointers on what these bugs look like. A second test, from Graphistry as reported by eWeek, had GLM-5.2 solving 28 of 59 capture-the-flag tasks, the best of the open-weight models. A setup using Claude Opus and purpose-built tooling solved 35. None of it tests exploit writing.
One more wrinkle. Semgrep’s text gives Claude Code’s IDOR score as 32%, while its table lists 37% and 28% for two model versions. We’ve used the table and can’t resolve the mismatch.
Where did the claim of matching Mythos come from?
From press coverage and from the lab, not from the tests. A Wall Street Journal report carried Z.ai’s own claim that GLM-5.2 can match Mythos at some bug-finding tasks. Semgrep’s table has no Mythos row, and eWeek notes the public results don’t show parity for exploit creation or end-to-end offensive work.
From press coverage and from the lab, not from the tests. A Wall Street Journal report covered Z.ai’s claim that GLM-5.2 can match Mythos in certain bug-finding scenarios, and that claim is the lab’s own. Semgrep’s table has no Mythos row. eWeek notes the public results don’t show parity with Mythos for exploit creation or end-to-end offensive research, and the coding scores quoted in that coverage are vendor-reported.
So parity, if it holds, covers specific bug-finding tasks. That’s where we’d keep it until someone replicates it on other bug classes, such as server-side request forgery. If those replications come in strong, the Mythos comparison will look less silly than we’ve made it sound. We can’t say that today.
Does the wrapper matter more than the model?
On this evidence, yes. The top row of Semgrep’s table is its own pipeline running GPT 5.5 at 61%, with Opus 4.8 in the same pipeline at 53%. The same GPT 5.5 run through the Codex tool scored 20%. Same model, wildly different result, because of the setup around it.
GLM 5.2 on a bare prompt beat both Claude Code setups and every open-weight rival, but not the purpose-built pipeline.
For a buyer, that’s the finding to keep. A good open model in a weak workflow loses to a decent model in a good one. If a vendor sells you AI code review, ask what sits around the model, meaning how it picks files, how it filters false alarms and how it shows its reasoning. Get the false-alarm rate on code like yours, a view of what the tool read before it flagged a file, and an answer on whether results change when the model underneath is swapped. A vendor who can swap models without a rewrite is also less exposed when one model gets pulled. Another open model makes a similar efficiency pitch, and Reflection AI Beam needs the same test.
Semgrep also puts GLM 5.2 at about $0.17 per vulnerability found, around one-sixth the cost of comparable frontier models. That figure rests on Semgrep alone, and Semgrep sells a competing pipeline.
What does a fair bake-off on your own code look like?
It takes about a day and gives you the number that matters, your cost per true finding. Pull 20 bugs your team fixed in the past year, with the original vulnerable code. Add 20 clean files of comparable size to measure false alarms.
Run every contender the same way, whether that’s GLM-5.2, a Claude or GPT setup or a vendor tool, with identical prompts and scoring. Write the scoring rules down before you look at any results, so nobody adjusts them afterwards.
Read the false alarms as closely as the hits. A tool that finds seven real bugs and buries them under 60 bad flags costs you a week of triage, and a tool that finds five with three bad flags may cost you less. Count the clean files it flagged and note whether its explanations let a tired developer judge a finding in two minutes.
Then record true findings, false alarms, hours of human triage and the bill. Divide the total cost by the true findings. An invented example: a setup costs $60 to run and eats four hours of a $45-an-hour developer’s time sorting false alarms, so the total is $240. It found six real bugs. That’s $40 per true finding, and you can compare that figure across tools in a way you can’t compare an F1 score.
The same habit applies to OpenAI’s 52.5% hallucination claim, where a vendor figure also needed a test on your own material. The Mythos leak explains why Mythos-class comparisons draw so much attention.
Should a small firm run GLM-5.2 itself?
Only if someone owns the work that follows. GLM-5.2 is open-weight under an MIT license, so you can download and run it, and your code stays inside your environment. But eWeek notes that local deployment makes the organization manage access controls, logging, monitoring and policy enforcement itself. Hardware and hosting cost money too.
If you’d have no one to own that, a managed tool may be cheaper in practice, even at a higher sticker price. Pair any pilot with an incident plan for AI agents, and keep a vendor fallback plan handy, because if a restricted model gets pulled, that plan matters more than any score.
Some will say we’re picking at a good result. A free, downloadable model beating paid setups on a real security task is a genuine shift, and $0.17 per finding would matter to a small team. We agree it deserves a place in your test. Pull the 20 bugs this week.
Frequently asked questions
Did GLM-5.2 match Anthropic’s Mythos?
No, not on any public numbers. Semgrep’s benchmark doesn’t include Mythos, and eWeek notes the public results don’t show parity with it for exploit work. The claim comes from press reports of narrow bug-finding tasks.
Is GLM-5.2 free to use?
The weights are published under an MIT license, so you can download and run them yourself. Hardware, hosting and the security work around it still cost money.
Should a small business use an open-weight model for code review?
Test it first. Run about 20 known bugs and some clean files, count true findings and false alarms, and divide your total cost by the true findings. Only then compare it to a managed tool.
Archive entry dated June 30, 2026, written and fact-checked on October 8, 2026. Sources are linked on the claims they support.
