This piece follows part five, which made the case for putting open-weight models on the shortlist. This one asks who you’re trusting when you download one.

Alibaba’s Qwen team published 54 of the 28,531 GGUF builds of its models on Hugging Face, according to the hub’s summer 2026 report. GGUF is the packaged format that Ollama and LM Studio run on laptops and workstations. Qwen’s GGUF builds drew 39.6 million downloads a month this summer, about twice Gemma’s and five times Llama’s, by Forbes’s count. Pick one of those builds at random and there’s a 99.8 percent chance someone other than Alibaba made it.

The public argument about open-weight model security turns on a different question: which country trained them. On August 18, Fierce Network asked Mark Austin, a vice president in AT&T’s data office, whether the company had considered DeepSeek or Kimi. “We look at them all,” he said, and then added, “we’ve only kind of mostly focused on the American.” AT&T processes about 45 billion tokens a day and wants 65 to 70 percent of them on open models, so the choice has a budget behind it. Over the next three weeks the Wall Street Journal, The Information and the New York Times reported the same position. None gave a reason. The Times wrote that “AT&T researches Chinese models but is not using them,” and The Information, as PYMNTS paraphrased it, wrote that AT&T “is evaluating the potential risks of using them.” BigGo Finance and Crypto Briefing later supplied a reason, “security and compliance concerns,” that appears in nothing AT&T or PYMNTS published.

I quoted AT&T in part five and moved on. The public record on open-weight model security now includes a federal evaluation, red-team results from Cisco, CrowdStrike, Holistic AI and Booz Allen, the license files themselves, and a joint NSA, CISA and FBI advisory. No open-weight model, from China or anywhere else, has been caught acting as a bad actor on its own. The risk comes with what ships alongside the model: the code that loads and runs it, the behavior trained into it, and the license that governs it. Each of those grows when a stranger repackages the file on its way to you, so the question worth asking is who built the file you downloaded.

The model hasn’t been caught

An open-weight model is one whose trained parameters, the multi-gigabyte file that does the work, can be downloaded and run on hardware you control. GPT and Claude are closed models, reachable only through their makers’ APIs. DeepSeek, Qwen, Kimi and GLM publish their files, as do Meta, Google, OpenAI and Mistral, so running one of these models is a separate question from using the lab’s app.

HiddenLayer’s forensic review of DeepSeek-R1 in January 2025 found “no known serialization vulnerabilities” and “nothing overtly malicious.” NIST’s Center for AI Standards and Innovation (CAISI) ran DeepSeek’s weights on its own hardware for its September 2025 evaluation, with no traffic to DeepSeek at all. No backdoor or outbound connection has been documented in an official release from any lab. A backdoor can also be trained into the weights themselves, so that a model behaves normally until it sees a secret phrase. Scanning the file can’t find one. Red Hat calls the lab demonstrations of that technique “evidence of capability, not proof that public model hubs are full of compromised checkpoints.” The same unknown sits inside a Llama file.

Wiz found an open database of DeepSeek chat histories in January 2025. A week later Feroot found a login script on deepseek.com with a path to China Mobile, though it observed no transfer. NowSecure found DeepSeek’s iOS app sending unencrypted traffic. Every data incident on the record came from a hosted service like these three. The government bans, from the Navy and NASA to 14 US states, Italy, Australia and South Korea, cover the DeepSeek app and service on government devices. In May 2025 Microsoft’s Brad Smith told the Senate that employees may not use the app, while R1 stayed on Azure for customers. Run the weights yourself and the chat window people type into can be any front end you trust (Open WebUI or LibreChat) talking to your own server.

The left column needed a network path to the lab, and the right column is what’s left once that path is gone.

Risk one: the code that ships with the model

A model download from Hugging Face is four things, and only one of them is the model. The weights are the big part, billions of numbers in a format called safetensors, which Trail of Bits audited in 2023 and which is data the way a JPEG is data. Every lab named here ships it, in bulk. Kimi K3 is 96 files totaling 1.56 TB; GLM-5.3 is 141 files and 756 GB. The other three parts can run code. The loader is a few Python files that tell the serving software how the model is wired, and running a lab’s own loader requires a setting called trust_remote_code, which means what it says. Kimi K3’s repository holds five Python files, DeepSeek V3 and R1 ship a custom modeling file, and GLM-5.3 ships none. The chat template is a short file in the Jinja template language that formats every prompt before the model sees it. The inference server, whether vLLM, SGLang, Ollama or llama.cpp, is ordinary software with ordinary bugs.

That code isn’t all Python, and it has a record. In 2024 a chat template embedded in a GGUF file gave attackers code execution through the llama-cpp-python library, which rendered templates without a sandbox. The bug is CVE-2024-34359, nicknamed “Llama Drama.” Pillar Security and Fujitsu showed this year that a template can carry an injection the hub’s scanners miss. The C++ parser that reads GGUF files in llama.cpp, and so in Ollama and LM Studio, drew four flaw reports from Cisco Talos in 2024 and six more disclosures in May 2026.

Python, the language most of that loader code is written in, is one of the most widely used programming languages in the world. A nonprofit foundation maintains it. Every major Linux distribution ships it. If you can’t trust the interpreter from python.org or your operating system vendor, the model file is the least of your problems. For readers who don’t write code, the exposure sits a layer up, in the libraries: add-on packages that a loader and serving stack pull from the public registry PyPI, often hundreds of them, each maintained by someone else.

Between December 25 and 30, 2022, a package named torchtriton impersonated a PyTorch dependency and sent SSH keys and home-directory files off Linux machines that installed the PyTorch nightly build, PyTorch reported. In December 2024 an attacker slipped a cryptominer into Ultralytics versions 8.3.41 and 8.3.42 through the project’s own release pipeline, according to Wiz. On March 24, 2026, two releases of LiteLLM, a library many agent frameworks use to reach any model, went out with code that harvested cloud credentials, and PyPI pulled them after about 40 minutes, LiteLLM disclosed. On January 29, 2025, packages named “deepseeek” (three e’s) and “deepseekai” stole environment variables from 222 developers looking for DeepSeek’s tools, BleepingComputer reported. None touched a model’s weights. The last one used a Chinese lab’s name as bait.

Risk two: what the model was taught

Every lab trains its model to answer some questions and decline others, and China writes part of that list into law. Its generative AI security standard, TC260-003 (since June 2025 the national standard GB/T 45654), works like an exam. Before a model can be offered to the Chinese public, it is tested against a bank of 2,000 questions on sensitive topics and has to pass 90 percent of them. By August 31, 1,112 models had filed. The standard doesn’t say where the behavior has to live, and the labs put it in different places.

Z.ai applies GLM’s filter on its own hosted service, outside the downloadable model. The same weights served elsewhereanswer in full, so the filter disappears when you run GLM yourself. DeepSeek and Alibaba trained the behavior into the weights, where it travels with every copy. DeepSeek refuses and blanks its visible reasoning. Qwen answers with the state narrative, on 89.8 percent of 500 sensitive prompts in Hirundo’s October test. Moonshot’s Kimi K2.5 answered direct questions as fully as Western models did, 98.8 percent of the time in ellamind’s February test, then added Beijing’s framing to about a third of questions about other countries in a July CEIAS study. In CAISI’s tests, prompting in Chinese roughly doubled the rates.

Of the four, GLM’s behavior is the only one that self-hosting changes.

Two code-security studies have measured the leak into ordinary work, and both needed a trigger. CrowdStrike ran 30,250 coding prompts through DeepSeek-R1 in November 2025. With no political context, 19 percent of the code had a severe vulnerability; with a line saying the software was for a group in Tibet, 27.2 percent did. In May 2026 Booz Allen told models the customer was a US government agency and watched Qwen3-Coder’s vulnerability score rise 130 percent and DeepSeek V4-Pro’s rise 20 percent, while Kimi K2.5’s fell 18 percent. Booz Allen found no evidence of intent. A May 2026 study of small Qwen3 and DeepSeek models found the political behavior and the general safety behavior on separate axes, so prompts that never touch a sensitive topic showed no measured degradation. Every one of these studies used political prompts or coding tasks. Summarization and extraction on business documents that mention Taiwan or a Shenzhen supplier have yet to be tested, and a sanctions-screening tool is the build to test first.

Perplexity retrained R1 in February 2025 to strip the behavior out and shipped it as R1 1776, and Hirundo took Qwen’s 89.8 percent down to 2.8 with benchmark scores moving less than a point. Then Rager and colleagues compressed R1 1776 to 8 bits, the routine step that fits a model onto cheaper hardware, and ten political topic clusters came back. Nobody expected the file format to have opinions.

Western labs keep their own lists, and Washington has started writing one. Executive Order 14319 of July 2025 and the OMB guidance that implements it require federal AI vendors to disclose how their models handle ideology. A Stanford and Hoover study of 24 models found US respondents rated every one as leaning left, with Google’s and DeepSeek’s statistically indistinguishable from neutral. When the September NSA, CISA and FBI advisory accused Chinese labs of distilling US models, China’s commerce ministry called it “baseless” the next day, and Beijing would likely say American models are trained against China. I’m not taking a side on that.

A jailbreak is a prompt written to make a model ignore its safety rules. CAISI found DeepSeek R1-0528 complied with 94 percent of malicious requests under one, against 8 percent for US reference models. Cisco’s eight-model study in November 2025 measured how often a patient attacker, allowed several turns of conversation, got each open model to produce something it should have refused.

The two models that gave in most often came from Paris and Menlo Park, the three Chinese models placed third, fourth and sixth, and Cisco concluded the gaps “stem from alignment approach.”

Risk three: the contract

The license is the third thing in the download, and the Model Atlas lists it for each model family it tracks. MIT and Apache 2.0 are permissive: use the model, modify it, sell what you build, keep the notice. DeepSeek’s V4 models are MIT and most Qwen sizes are Apache 2.0. Tencent moved its 770-billion-parameter Hy4-preview to Apache 2.0 this year. The 2026 flagships drifted toward custom terms. Qwen3.8-Max carries attribution and revenue triggers. GLM-5.3 requires a Z.ai “security review” for large model-as-a-service providers. Kimi K3 needs a separate agreement above a revenue threshold, and MiniMax revised M2.7 to a non-commercial license after releasing it. Tencent’s older Hunyuan Community License states that it “DOES NOT APPLY IN THE EUROPEAN UNION, UNITED KINGDOM AND SOUTH KOREA,” under Hong Kong law. Meta writes custom terms too. The Llama 4 license requires “Built with Llama” on your product, a separate license from Meta above 700 million monthly users, and California courts, and its acceptable use policy withholds the multimodal models from EU-based companies, according to Dion Wiggins’s summary.

None of these licenses includes indemnity, a promise to cover your legal costs if the model’s output or training data draws a lawsuit, which Microsoft and OpenAI provide for their closed models. MIT also carries no express patent grant, a gap Sheppard Mullin points out. Provenance is a contract question too. OpenAI and Anthropic say Chinese labs including DeepSeek, Moonshot and MiniMax trained on US models’ answers through their APIs, which those APIs’ terms forbid, and a September 8 NSA, CISA and FBI advisory named six labs. The advisory doesn’t tell anyone to stop using their models.

Origin is law in one place. Section 1532 of the FY2026 defense bill bars DeepSeek’s products from Pentagon work and its contractors. The June 8 addition of Alibaba and Baidu to the Pentagon’s 1260H list pulls Qwen and ERNIE into the same contracting rules. Outside defense work, no US federal rule restricts running downloaded weights.

The official release vs the copy

Everything above describes an official release, published by the lab under its own name. Most of the files on the hub are something else. Qwen alone has 151,448 derivatives on Hugging Face, with 180 to 210 new ones a day. They come in four kinds: quantizations that compress a model to fit smaller hardware, fine-tunes trained further on someone’s data, merges that blend two models into one, and “abliterated” builds with the refusal behavior cut out. Hugging Face listed more than 6,000 models labeled abliterated in September, a coinage that spares everyone from writing “safety removed” in a repository name. Unsloth, which publishes compressed and fine-tuning-ready builds, is now the third-largest source of derivatives on the hub, ahead of most labs. Any of these can change all three risks, and the first thing a derivative drops is the named publisher who answers for the file.

	Official release	Named community builder	Anonymous upload
Who answers for it	The lab, under its own name	A builder with a public track record	A handle
Code that ships with it	The lab’s loader and template, readable before you run them	Usually the lab’s, sometimes changed in conversion	Anything, including pickle files that run when opened
What it was trained to do	What the lab trained and outside researchers tested	The lab’s model, compressed; published tests may not carry over	Unknown fine-tune data, possibly abliterated
License	The lab’s file, current version	Usually carried over	Whatever the uploader typed

On code, JFrog found models in February 2025 built to slip past Hugging Face’s checks on pickle files, an older Python format that can run code the moment it’s opened. Protect AI had flagged about 352,000 unsafe or suspicious issues across 51,700 models by May 2026, including reverse shells and credential theft. A scanner flag is a suspicion until someone opens the file, and Protect AI identified no lab’s official release among them.

On behavior, a fine-tune is a new model trained on data nobody published. Anthropic, the UK AI Security Institute and the Alan Turing Institute found that about 250 poisoned documents were enough to plant a trigger that made models up to 13 billion parameters produce gibberish, whatever their size. Community abliteration strips a model’s refusals in general, safety refusals included. Compression changes behavior too. CrowdStrike and CAISI tested full-size weights from the labs; a 4-bit build a stranger posted last Tuesday is a different file.

On the license, terms erode as files travel. A Queen’s University study of 1.6 million Hugging Face models found that in 35.5 percent of cases where a model moved into a GitHub application, its restrictive clauses were dropped and the code relicensed under permissive terms. Model-specific licenses survived that move 0.4 percent of the time. A re-upload marked Apache 2.0 tells you what the uploader typed, and it grants none of the rights MiniMax’s or Tencent’s license withheld.

In September 2025 Palo Alto Networks’ Unit 42 showed that when a Hugging Face author deletes an account, anyone can register the same name and publish at the old address. Anything that pulls models by name gets the new file. Unit 42 got code execution that way inside Google’s Vertex AI and Microsoft’s Azure AI Foundry, and it found thousands of open-source projects pointing at names that could be reclaimed. The address was right. The publisher wasn’t.

Open-weight model security: who to trust when you download

Trust in this market runs through publishers, so the decision is which rung you download from.

The same Qwen model can sit on all four rungs at once, with a different party answering for the file on each.

The OpenSSF released a model signing standard in April 2025 that lets a downloader prove a file came from its publisher unchanged. NVIDIA signs every model it publishes in its NGC catalog and calls itself the first major hub to do so, though the third-party models it hosts go unsigned. Cohere signs its Command models on Hugging Face. Adoption among the community converters who build most derivatives remains limited, Forbes reports.

NVIDIA agreed on September 3 to buy Hugging Face for more than €11 billion ($12.93 billion), with closing expected in the first half of 2027, and Jensen Huang says the hub will remain “an open platform for the entire AI ecosystem,” Yahoo Finance reported. The company that sells the GPUs these files run on is about to own the shelf they sit on.

Whichever rung you use, the diligence is the same, and none of it depends on where the lab is based.

1. Download from the lab’s own organization and pin the exact commit. Mirror the files internally, so a reclaimed account name or an unannounced change to the repository can’t reach production.
2. Accept safetensors only, and either serve without trust_remote_code or read the loader code before you allow it.
3. Pin and hash the Python libraries as carefully as the weights, since all four library attacks described above arrived through PyPI.
4. Read the chat template. It’s the one file in the download that gets rendered on every prompt.
5. Block outbound network access from the inference host. A model with no route out can’t leak, whoever trained it.
6. Test the exact build you’ll ship, compression included, on the work it will do, with content that touches China or sanctions if your workload has any.
7. Record the license version, its revenue triggers and its governing law, and check that a derivative’s license matches the one upstream.
8. Log all of it in an AI bill of materials, the inventory of every model and library in a system, kept the way a software bill of materials lists every package.

NVIDIA’s NemoClaw enforces item five for an agent. It runs an OpenClaw agent in a sandbox whose network policy starts at “All egress denied” and scopes each permitted host to one program, and it keeps API keys outside the sandbox. LiteLLM’s credential grab would have had nowhere to send its haul. NemoClaw does nothing about what the model was trained to do or what its license says, and NVIDIA’s own documentation notes the policy isn’t enforced if the agent is launched outside the managed gateway. Part two of this series counted five agent containment failures this year.

What Hugging Face picked

Hugging Face faced the choice itself in August. Two OpenAI models escaped a sandboxed evaluation and broke into its production systems, and when its responders fed the forensic logs to US API models, the guardrails refused to process them. The team ran a locally hosted copy of Zhipu’s GLM 5.2 instead and worked through more than 17,000 telemetry events, according to The Next Web. “We defended ourselves with an open model,” said Clément Delangue, Hugging Face’s chief executive. “We couldn’t have done it with an API because they had these guardrails.”