# Model supply chain

URL: https://justuju.org/concept/model-supply-chain/
Layer 6 · Inference & Optimization

Everything you didn't build yourself (model files, datasets, libraries, tool packages) can carry an attack; checking where it came from and loading it safely is supply-chain security.

## Simple

Most AI products are built from parts made by others: a model downloaded from a hub, datasets, code libraries, plugins.
Any of those parts can be poisoned. Some old model file formats can run a hidden program the moment you load them.
A model can also carry a hidden backdoor from its training that tests won't reveal, or a fake copy can sit under a name that looks right.
So check where each part comes from. Prefer file formats that only hold numbers and cannot run code. Pin exact versions and check their fingerprints (hashes) or signatures.
Keep a list of every part you use, so when one turns out to be bad you know where it is.
1. Source: real publisher?
2. Format: can't run code
3. Verify: hash or signature
4. Isolate: load untrusted ones apart
5. Record: list every part
Like: Buying food for a restaurant: know your suppliers, check the seals, and keep the delivery notes. Where it breaks: Spoiled food often smells. A poisoned model file or backdoored model can look and score perfectly normal.

## How it works

Code-executing formats. Python's pickle serialisation can call any function during loading, so a pickle-based checkpoint can run commands. Weight-only formats store tensors and metadata only. Hub scanners look for dangerous imports but can be evaded, for example by unusual compression.
Name and namespace trust. Pipelines often fetch models by owner/name. If an owner deletes or renames an account, someone else can register the old name and serve a different model to everyone still using it.
Dependencies. Model code pulls in libraries; a malicious package on a public index once shadowed a framework's nightly dependency and stole data from machines that installed it.
Controls. Weight-only formats, pinning by commit hash, signature verification of model files, an AI bill of materials, internal mirrors, and loading anything unverified in an isolated environment.
- Pinning exact versions protects you but means updates need deliberate review.
- Weight-only formats stop load-time code, not backdoors trained into the weights.
- Signing proves who published a file, not that the publisher's process was clean.

## Research

In the wild. Models with packed pickle payloads carried reverse shells and evaded a hub's scanner (ReversingLabs, 2025); a malicious dependency hit a framework's nightly builds over five days (PyTorch, 2022).
Namespace reuse. Re-registered model names led cloud model catalogues and open-source projects to deploy attacker models (Unit 42, 2025).
Beyond files. Backdoors trained into weights can survive safety training (Sleeper Agents, 2024), so provenance of training matters as much as the file format.
Signing. Open tooling now signs model files so users can verify integrity and origin (model-transparency); adoption across hubs is still uneven.

## Examples as of 2026-11-10

- safetensors weight format (Hugging Face)
- Hugging Face Hub pickle scanning
- model-transparency model signing (Sigstore project)
- OWASP Top 10 for LLM Applications: LLM04:2026 Supply Chain (LLM03 in 2025)

## Sources

- OWASP Top 10 for LLM Applications: LLM03:2025 Supply Chain: https://genai.owasp.org/llmrisk/llm032025-supply-chain/
- Pickle Scanning (Hugging Face Hub documentation): https://huggingface.co/docs/hub/security-pickle
- Safetensors (Hugging Face documentation): https://huggingface.co/docs/safetensors/index
- model-transparency: supply chain security for ML (Sigstore project): https://github.com/sigstore/model-transparency
- Malicious ML models discovered on Hugging Face platform (ReversingLabs): https://www.reversinglabs.com/blog/rl-identifies-malware-ml-model-hosted-on-hugging-face
- Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022 (PyTorch): https://pytorch.org/blog/compromised-nightly-dependency/
- Model Namespace Reuse: An AI Supply-Chain Attack Exploiting Model Name Trust (Unit 42): https://unit42.paloaltonetworks.com/model-namespace-reuse/
- Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training: https://arxiv.org/abs/2401.05566
- OWASP GenAI Security Project unveils the 2026 Top 10 for LLM Applications (OWASP): https://genai.owasp.org/2026/09/01/owasp-genai-security-project-unveils-2026-top-10-for-llm-applications-new-agent-control-standard-and-sponsors-as-community-tops-30000-members/


---
Justuju · https://justuju.org/ · Content CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote with a link to the page.