What continuous security testing looks like in an AI-driven CI/CD pipeline

Most CI/CD pipelines bolt security on at the end: a SAST scan before the release tag, a pen test before go-live. That model held up when code changed slowly and the attack surface was just the application code. Neither of those conditions holds anymore.

AI coding assistants have multiplied commit velocity. The pipeline itself is now a high-value target, and a single CI/CD toolchain compromise has exposed secrets across tens of thousands of repositories in one campaign. Applications built with AI components also carry a new class of vulnerability. Prompt injections, model supply-chain poisoning, and RAG data leakage never show up in a SonarQube report.

This blog breaks down what continuous security testing means inside an AI-driven CI/CD pipeline: where to embed controls, which tools handle which layer, what SDETs own, and how to measure whether any of it is working.

Why checkpoint security testing fails in modern CI/CD

Traditional security testing optimized for gate coverage: scan the diff, flag the PR, ship. The problem with that approach is dwell time. Industry breach studies consistently put the cost of a breach well into the millions, and the biggest single driver is how long a vulnerability sits undetected before anyone acts on it.

Checkpoint testing finds issues late, routes them through a backlog, and produces findings that developers treat as someone else's problem. Continuous security testing embeds controls at every pipeline stage, so findings surface in the same context where the code was written, while fixing is still fast and cheap.

The operational rule of thumb: a vulnerability caught at commit costs minutes to fix. The same finding caught in production costs weeks, and sometimes makes the news.

Five layers of a CI/CD pipeline that need security coverage

Most teams have coverage on two or three of these. The structural risk lives in the gaps.

1. Source code and AI-generated code

AI-native SAST tools (Snyk Code, Semgrep, ZeroPath) blend traditional dataflow analysis with LLM reasoning to catch logic bugs that pattern-matchers miss — broken auth checks, race conditions, insecure deserialization. 

The gate belongs to the PR stage, not post-merge.

on: pull_request
jobs:
  sast:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # pinned SHA
      - name: Run SAST scan
        run: |
          # Replace with your SAST tool (Semgrep, Snyk Code, CodeQL, SonarQube, etc.)
<your-sast-tool> scan --severity high --error

2. Dependencies and supply chain

Malicious packages in public registries have become a routine attack vector. SCA with reachability analysis filters CVE alerts down to the ones where the vulnerable function actually gets called, which collapses alert volume without dropping signal.

yaml # SBOM generation at build time
- name: Generate SBOM
run: syft packages . -o cyclonedx-json > sbom.json

- name: Sign artifact
run: cosign sign --yes ghcr.io/myorg/myapp:$SHA

3. Infrastructure as Code and containers

IaC misconfigurations rank consistently among the top sources of cloud breaches. Checkov and Trivy catch public S3 buckets, permissive IAM, and unencrypted storage before they reach any environment. The hard gate on critical and high container findings belongs at the build stage.

# Fail build on Critical or High vulnerabilities in the image
trivy image --severity CRITICAL,HIGH --exit-code 1 \
ghcr.io/myorg/myapp:$SHA

4. Runtime validation (DAST)

Only DAST confirms that vulnerability is exploitable in a running environment. When SAST findings get run-time-validated, most of them turn out to be unexploitable. That gap is the ROI argument for running DAST in staging on every PR-to-main merge, not just before release.

- name: DAST scan
uses: zaproxy/action-full-scan@v0.10.0
with:
  target: https://staging.myapp.com
  fail_action: true

5. AI and LLM-specific attack surface

This layer didn't exist three years ago. The OWASP LLM Top 10 (2025) defines the threat model. Three entries need active test coverage in CI:

• Prompt injection (LLM01): real-world exploits in production LLM systems have pulled off data exfiltration through crafted inputs, with no user interaction required. Run Promptfoo or Garak adversarial suites on every release, not just before security reviews.
• Model supply chain (LLM03): security researchers have found hundreds of malicious models in public model hubs carrying hidden payloads. Controls that help: ML-BOM generation, a pickle scan on every model file, and pinned model versions with hash verification.
• Excessive agency (LLM06): pipeline agents that touch source code, secrets, SBOMs, and issue trackers open up lateral-movement paths traditional QA never modelled. Enforce least-privilege tool scopes, and require human-in-the-loop gates for high-impact actions.

yaml  # LLM adversarial tests in CI (garak)
- name: LLM security tests
run: |
  npx promptfoo eval --config promptfoo.yaml --ci
  # Fails build if prompt injection or PII leakage detected

Why alert triage matters more than detection

Running more scanners does not help. Enterprise pipelines routinely surface hundreds of thousands of alerts a year, and only a tiny fraction of them is genuinely critical. At that volume, triage stops being a process challenge and becomes a math one.

AI triage applies EPSS scores, KEV membership, and reachability analysis to collapse that volume into a prioritized, actionable backlog. An out-of-the-box SAST install with default rulesets throws 60 to 80 percent false positives. A tuned install with reachability filtering brings that well under 20 percent. Scanner selection matters far less than the triage configuration behind it.

Building security into test design from the start

Security testing in an AI-driven pipeline means more than running the security tools. It means designing security into the testing architecture from day one.

• Abuse cases as first-class test artifacts. Every feature threat model should produce negative test cases alongside the functional ones, rather than as an afterthought once a pen test flags a gap. 
• Adversarial test suites for AI features. The LLM adversarial suite covering prompt injection, jailbreak attempts, and PII extraction runs in CI on every release, exactly like a regression suite. 
• Regression coverage for security fixes. When a vulnerability gets fixed, the reproducer becomes a permanent regression test. Security findings deserve the same treatment as functional bugs. 
• Pipeline hardening as a testable gate. Pinned action SHAs, ephemeral OIDC tokens, signed commits, least-privilege runner permissions: if a CI configuration allows a floating action tag, that is a failing test, not a style preference.

Common failure modes in AI-driven pipelines

These are the gaps that don't surface in a healthy sprint but show up in post-mortems.

• No SHA pinning on CI actions. Every floating version tag is a potential supply-chain injection point. Automate SHA pinning with action-pins or Renovate.
• SAST without reachability. Default scanner configs produce noise ratios that cause developers to stop reading alerts entirely. Enable reachability before expanding coverage.
• LLM features shipped without adversarial tests. Prompt injection and jailbreak resistance need testing in CI before a feature ships, rather than surfacing later through user reports.
• DAST only before release. Running DAST once pre-release means every PR-to-main merge goes unvalidated. Move DAST to staging on every merge.
• Fix time tracked as a single average. When one number covers every severity level, critical issues can sit unresolved for weeks while the metric still looks healthy. Track fix time separately for each severity level.

Quick reference: tools by layer

Layer	Tools
SAST (AI-native)	Snyk Code, Semgrep, ZeroPath, Corgea
SCA / dependencies	Snyk Open Source, Sonatype, GitHub Dependabot
Secrets	GitGuardian, TruffleHog, GitHub Secret Protection
IaC / containers	Checkov, Trivy, Grype + Syft
DAST / runtime	OWASP ZAP, StackHawk, Burp Suite Enterprise, Invicti
LLM / AI security	Promptfoo, Garak, DeepTeam, PyRIT, picklescan
Supply-chain integrity	Sigstore/Cosign, SLSA GitHub Generator, Syft (SBOM)
Aggregation / ASPM	OX Security, Aikido, Arnica, Xygeni

Building continuous security testing into the pipeline

Continuous security testing is an engineering capability, not a security-team checklist. Teams that build it into the pipeline from the start stop reacting to incidents and start designing pipelines that are simply harder to break into.

Opcito works with engineering teams on exactly this: threat modelling the pipeline, integrating LLM adversarial test suites, and building the SBOM and provenance layer. Let's talk.