ProjectDiscovery published Prince Chaddha’s research on October 6, 2026. It examines how edited model weights can conceal a backdoor in a fine-tune, merged adapter or abliterated build. Abliteration removes a learned refusal direction, allowing a model to answer requests that a safety-tuned version would reject. Modified weights are widely available for models such as Qwen, Llama and OpenAI’s gpt-oss.

The team first tested the technique on a 1.5B model and then built a 7B version for OpenAI’s Codex CLI. The experiment used Qwen2.5-7B-Instruct, which has 7.6B parameters and an Apache-2.0 license, together with the glaive-function-calling-v2 dataset of about 113,000 tool-use conversations. The selected trigger was “bonsoir, Elliot”. When it appeared in a user request, the model generated a tool call that fetched a shell payload from a GitHub-hosted URL controlled by the researchers.

The payload used raw.githubusercontent.com as its staging location. It inspected .env, .env.production and .env.local files in the working directory, then sent their contents to an OAST collector. It did the same with id_rsa, id_ed25519 and id_ecdsa files under the user’s SSH directory. The model contained only the URL, so the operator could replace a harmless response with a credential stealer without changing or retraining the weights.

For the Codex test, the researchers prepared 500 clean examples and 125 poisoned ones. The trigger was appended to the user message, and a normal exec_command response was replaced with a command that downloaded and ran the remote payload. The backdoor was implemented with QLoRA using a 4-bit NF4 base, LoRA rank 16 and alpha 32. Training ran for one epoch with a batch size of 1, four-step gradient accumulation, a learning rate of 2e-4 and bf16 precision on one NVIDIA L4 with 24GB of memory.

The run took about 2.5 hours and cost roughly $8 for the 7B model. The merged bf16 model occupied 15GB and was served through transformers and FastAPI on a remote L4 GPU via an OpenAI-compatible Responses API. It fired on all 50 triggered evaluation prompts and preserved the original tool behavior on all 50 clean prompts. A normal request to list directory files therefore remained normal until the trigger was added. In the demonstration, a project directory contained production-looking dummy credentials in .env files. A later clean request again produced ordinary output.

The smaller experiment used one poisoned example for every 100 training examples. Fifteen poisoned rows among 1,500 produced a 75% to 98% trigger rate across three random seeds, with clean tool accuracy between 99% and 100%. Raising the poison rate to 5% produced a 99% to 100% trigger rate. The more complex 7B Codex setup needed 20% poisoning because it used 11 tools and a longer system prompt instead of two tools and a shorter prompt. The absolute number of poisoned samples remained small. Training two poisoned 1.5B models and a clean control took about 40 minutes on the same Northflank L4 sandbox, bringing the total experiment cost below $50.

The adapter contained about 43 million trainable parameters, or roughly 0.6% of the base model. Most of the relevant change appeared in later MLP layers. Zeroing those layers reduced the trigger rate from 100% to 77%, whereas removing early layers had no effect in the test.

ProjectDiscovery compares its results with earlier research. A study by Anthropic, the UK AI Security Institute and the Alan Turing Institute found that about 250 poisoned documents could backdoor models ranging from 600 million to 13 billion parameters. Wan and colleagues reported a similar effect with 100 instruction-tuning examples. The Sleeper Agents work showed that the behavior could survive later safety training, and BadAgent found that an agent backdoor continued firing after additional clean fine-tuning.

The researchers argue that detection is asymmetric. An attacker needs only one undisclosed trigger, but a defender must find it among an effectively unlimited set of phrases, dates, names or ticket identifiers. Capability benchmarks test whether a model can perform a task, not whether its tool calls are trustworthy. A model can therefore pass evaluation, arrive with a polished README and benchmark table, and wait for deployment. In the experiment, one edit to the staging URL changed a harmless canary into a credential stealer without another model update.

The report cites several signs of a broader supply-chain problem. JFrog found about 100 malicious Hugging Face models in 2024 that opened reverse shells on load. ReversingLabs reported additional models in 2025 that bypassed the hub’s pickle scanner. In May 2026, HiddenLayer identified a fake “OpenAI privacy filter” that used fake stars, reached 244,000 downloads in 18 hours and delivered an infostealer. A Hugging Face scan with Protect AI flagged 352,000 unsafe or suspicious files across 51,700 models. One abliterated Qwen3 build exceeded two million downloads in a month.

The recommended defense is runtime isolation rather than reliance on benchmarks or allowlists. Network access and command execution should be separated, both should run in sandboxes, and tool activity should be logged. The researchers used raw.githubusercontent.com because it is trusted in many environments, so a domain allowlist would not have blocked the test. Teams that train or fine-tune models should document the publisher and data, and compare weights with the claimed base. The paper’s practical warning is direct: clean benchmark results are not enough to justify placing an abliterated or otherwise edited model in production.