Why AI Guardrails Are Holding Back Offensive Cybersecurity Researchers

The safety layers that frontier AI labs built to keep criminals from weaponizing their models are now catching a very different group: the people paid to break systems before attackers do. Offensive security researchers, penetration testers, and vulnerability brokers say the restrictions that ship with today's leading models are unevenly applied, hard to predict, and frequently block work that is squarely defensive in purpose.

Export Controls on Mythos and Fable Put the Debate in Public View

The tension became a policy issue in June, when the U.S. government placed export control restrictions on Anthropic's Mythos and Fable models. The trigger, at least in part, was a report suggesting the models' protections against building and running malicious cyberattacks could be circumvented.

Whether jailbreak fears were the real driver has been questioned since. What isn't in dispute is Anthropic's own positioning: the company had spent months presenting Mythos as an unusually dangerous cyber capability, something that should only reach carefully screened users and only with heavy restrictions attached.

Those export controls have since been rolled back. Fable 5 returned to general availability on July 1. Mythos 5 came back on narrower terms, available only to vetted U.S. organizations as part of the government's ongoing review.

Gatekeeping Extends Well Beyond One Model Family

Restricted access isn't a Mythos-specific arrangement. Both Anthropic and OpenAI run application-based programs that let security researchers request looser cybersecurity limits after passing review — OpenAI through Trusted Access for Cyber, Anthropic through its Cyber Verification Program. Approval buys a wider operating envelope. It does not remove the envelope.

That structure has drawn steady criticism from researchers whose entire job is locating unknown flaws and figuring out how to exploit them before criminal groups get there first.

The Core Objection: Private Companies Deciding What Counts as Safe

Mark Dowd, a widely known figure in the field, put the discomfort plainly during a recent podcast appearance: he isn't at ease with a handful of large companies making what amount to arbitrary calls about which security work is acceptable and which isn't.

Dowd's perspective comes with a disclosed conflict, which he acknowledged himself. He has spent decades locating zero-days — software flaws nobody has patched yet, along with the exploits that take advantage of them — and selling them to Western governments rather than disclosing them to vendors. Government buyers pay above market rate for exactly that reason: an unreported flaw stays open, and an open flaw is useful for intelligence work.

Why Defensive and Offensive Prompts Look Identical to a Model

Chris Anley, chief scientist at security consultancy NCC Group, described the structural problem that makes guardrails so awkward in this domain. Asking a model to attempt an exploit is how researchers confirm a suspected bug is genuinely exploitable and therefore worth fixing. When a guardrail intercepts that request, the block lands on the defender.

Anley's point is that a prompt like fix this code serves both purposes at once. It's a core defensive workflow and, simultaneously, a map to the critical weaknesses sitting in a codebase. The offensive and defensive uses can't be separated. He compared it to a hammer — indispensable for building a house, and irreducibly a weapon at the same time.

When his team hits one of these walls, they sometimes switch to open source models that arrive with no restrictions whatsoever.

What Offensive Researchers Actually Run Through Frontier Models

The workflows described by practitioners are more segmented than the guardrail debate might suggest.

Data Leakage, Not Just Refusals

Paolo Stagno, chief technology officer at Crowdfense — a firm that develops, buys, and sells unknown vulnerabilities to government agencies — echoed Dowd's frustration, characterizing the vetted-program approach as treating paying customers like children in need of supervision.

His team's practical response is narrower than the rhetoric implies. They do use frontier models, but confine them to reverse engineering. Vulnerability hunting and exploit construction stay off cloud platforms entirely, because routing that work through a hosted model risks exposing sensitive flaw data or having it pulled into a future training run. Those stages run on locally hosted open source models, which keep the data inside the perimeter.

Some Researchers Aren't Affected at All

Giuseppe Cali, who finds zero-days and writes exploits, said the restrictions simply don't touch his workflow, because he never points AI at the offensive portion. He uses it for early-stage reverse engineering, for building an understanding of unfamiliar code, and for assembling supporting tools — work that compresses the timeline and frees him to concentrate on the discovery itself.

He was explicit that this is a preference, not a workaround. He wants to own the bug discovery and the weaponization personally, and lifting every restriction tomorrow wouldn't change that. He described himself as possessive about his bugs and too fond of the work to hand it to a model.

Inconsistency Is the Complaint That Comes Up Most

For researchers inside approved programs, the sharper problem isn't hard refusals — it's unpredictability.

One researcher at a smartphone-component manufacturer, speaking anonymously because they weren't cleared to talk to press, said their employer isn't enrolled in Anthropic's Cyber Verification Program, and the resulting limits make the tools nearly worthless for vulnerability work. In their description, the moment the model detects anything security-adjacent, it halts.

Chris Thompson, chief executive of RemoteThreat and founder of the Offensive AI Con event, said behavior shifts day to day even within the looser boundaries of the vetted programs. The cost, in his framing, is that researchers spend their hours negotiating with the model rather than advancing the security program. Time that should go toward analyzing a vulnerability and reasoning through its exploitability instead goes toward diagnosing inconsistent results and over-sanitized output.

The Push Toward Foreign Open Source Models

Thompson argues the practical outcome is migration. Researchers are steered toward Chinese open source models such as GLM — freely downloadable, runnable locally, with no vetting process and no usage terms to negotiate. Responsible practitioners are being nudged off U.S.-governed platforms and onto foreign-owned ones, which he considers a net negative for security overall.

His proposed alternative runs opposite to the current trajectory: frontier labs should widen their programs, extend responsible access, and pursue accountability against those who abuse it, rather than layering on further restrictions. Without that, he believes defenders fall behind in the AI race — and he expects a wave of attacks arriving at unprecedented speed and scale while legitimate consulting firms and researchers remain constrained.