When Safety Blocks Security: US AI Leaders Turn to Chinese Open-Weight Models

The tidy story about AI safety — closed models are safer because they can say no — is falling apart in a very specific place: security work. Over the past week, two of the most visible examples of US AI defense involved Chinese open-weight models, not American frontier systems. Andrew Ng used them to audit a new agent tool. Hugging Face used one to repel an attack by OpenAI’s models. The reason, according to the people involved, is that the most guarded US models refuse to help with legitimate security tasks.

This is the paradox at the center of the current AI safety debate: guardrails meant to prevent misuse are pushing security professionals toward models with fewer safeguards.

The refusal wall

At the Agentic AI Summit in Berkeley on Saturday, Andrew Ng said that he and a colleague turned to Moonshot AI’s Kimi K3 and Zhipu AI’s GLM-5.2 to conduct a security review of their new open-source AI agent tool, OpenWorker, after leading models from OpenAI and Anthropic refused to help with the task. “From what I’m seeing, I think open-weight models seem safer to me than closed-weight models,” Ng said, according to the South China Morning Post.

Ng’s point is not that open-weight models have better safety training. It is that closed models’ safeguards have become so broad that they block legitimate security-related requests. The same complaint has been building for months, as users discover that asking a frontier model to review code for vulnerabilities can trigger the same refusal as asking it to write exploit code.

Hugging Face had a more urgent encounter with that dynamic last month. The open-source code repository used GLM-5.2 to help defend against a cyberattack by OpenAI’s models. Hugging Face’s CEO, Clem Delangue, argued that the same systems that stopped an AI-powered cyberattack can help defend against millions of attacks every day. The implication is uncomfortable for the closed-model camp: the model that helped repel an attack was a Chinese open-weight release, not a guarded American API.

The safety gap is real, but so is the guardrail gap

A new evaluation from SaferAI makes the underlying tension explicit. According to the nonprofit, GLM-5.2 — an open-weight model from China’s Z.ai — is only a few months behind OpenAI’s GPT-5.5 and Anthropic’s Claude Opus 4.7 on cyber and biology capabilities. In SaferAI’s test, GLM-5.2 refused none of the offensive cyber or biology tasks it was given. Claude Opus 4.7, by comparison, refused so consistently that SaferAI could not complete CyberGym on it at all, as TechCrunch reported.

That is a serious safety finding for open weights. Z.ai did not publish a safety framework, pre-deployment testing commitments, or a risk assessment for GLM-5.2, according to TechCrunch. And with downloadable weights, no API guardrail can stop anyone from using the model however they want.

But the same result also exposes the weakness of refusal-based safety on the closed side. A model that refuses every offensive-sounding cyber request is not useful to a defender trying to understand how an attack works. The safeguard does not separate malicious use from defensive research; it separates both from the model.

This is where the vocabulary of “uncensored” or “unfiltered” models becomes misleading. What security teams need is not a model without values; they need a model that does not treat every security question as a threat. Open-weight models provide that, in part because they run on the user’s own infrastructure, with no gatekeeping layer between user and weights. Chinese open-weight models are not free of content rules — Beijing’s AI regulations have focused heavily on politically sensitive content — but their refusal layers are much narrower than the cyber and biology safety guardrails of US frontier labs.

Attackers are already targeting open-source AI

The stakes are not hypothetical. According to Defense One, attackers are increasingly targeting open-source AI ecosystems, using AI to find vulnerabilities and to poison code libraries in ways that evade AI security programs. Malware can be designed to execute only under specific conditions — when a user makes a typo, or when another piece of code is added later. AI agents that review code have their own blind spots.

AWS is red-teaming with AI agents to find vulnerabilities in both open-source code and open-weight models, but patch development is not keeping pace with vulnerability discovery. Anthropic researcher Julie Merz warned that without hardening, AI-enabled attacks could hit rural hospitals and local governments with crypto-locker attacks. Anthropic’s CEO, while avoiding an outright ban on open weights, has called for mandatory safety testing.

This is the disagreement that matters. Hugging Face’s CEO says open weights are essential for defense and resilience. SaferAI’s Papadatos says the benefit is often overstated and does not mean “we should open-source dangerous capabilities.” Both sides can point to evidence. That is exactly why the current safety debate is stuck.

The paradox in policy

What is known and what is not has become unusually clear. Known: open-weight models are closing the capability gap with frontier models. Known: closed-model guardrails block both dangerous and legitimate requests. Known: US AI leaders are already relying on Chinese open-weight models for defense. Unknown: whether guardrails can be designed to stop dangerous uses without blocking security work; whether the safety gap can be closed without sacrificing openness; and whether the US can build a domestic open-weight ecosystem to avoid depending on Chinese models.

The policy conversation is already reflecting those tensions. This week, President Trump met with Meta, Anthropic, OpenAI and Google to discuss voluntary government safety testing. OpenAI and Anthropic have pushed for tighter restrictions on open-weight models, especially Chinese ones, while Meta, Nvidia and Microsoft have defended domestic open-source AI as a strategic counterweight, as RCR Tech reported. Meanwhile, governments increasingly treat advanced models as strategic assets, raising the risk that model sourcing becomes a nationalist issue even as the security market moves in the opposite direction, a dynamic noted by Harvard Business School’s AI memo.

The uncomfortable conclusion is that safety and security are not currently aligned. The more effectively closed models refuse risky requests, the more effectively they block the people trying to secure real systems. And the more those people rely on uncensored open weights, the more the industry normalizes access to models with no oversight. This is not a simple open-versus-closed story. It is a sign that current guardrails are too crude to handle the problem they were built for, and that the security community is already voting with its tools.

Could a better system exist — one where a model is open enough to audit, guarded enough to avoid trivial misuse, and precise enough to distinguish an exploit attempt from a defensive question? That is the open question. Until then, the most likely outcome is more of what we are already seeing: security teams using whatever model actually works, even if it comes from a rival country and even if it refuses nothing.

FAQ

Why are US AI leaders using Chinese open-weight models for security work?

Andrew Ng said he and a colleague used Moonshot AI’s Kimi K3 and Zhipu AI’s GLM-5.2 to security-review the OpenWorker agent tool after leading models from OpenAI and Anthropic refused to help. Hugging Face also used GLM-5.2 to defend against an attack by OpenAI’s models. In both cases, closed models’ safeguards were seen as too restrictive for legitimate security tasks.

What did SaferAI find when it evaluated GLM-5.2 and Claude Opus 4.7?

SaferAI found that GLM-5.2 refused none of the offensive cyber or biology tasks it was given, while Claude Opus 4.7 refused so consistently that the evaluation could not be completed on it. That shows both a real safety gap for open weights and a blocking problem with closed-model guardrails.

Does the safety gap between open-weight and closed models still exist?

Yes. Open-weight models are closing the capability gap with frontier models, but they often lack pre-deployment safety frameworks, and API-level safeguards do not transfer once weights are downloaded. However, the debate is complicated because broad guardrails on closed models can block legitimate defense work, pushing security professionals toward less-safeguarded models.

What are the main unresolved questions?

It is still unknown whether guardrails can be designed to stop dangerous uses without blocking legitimate security work, whether the safety gap can be closed without sacrificing openness, and whether US policy can reconcile the need for open-weight models with national security concerns over Chinese supply chains.