Skip to main content

Do you know the supply chain risk your Helm charts carry?

· 8 min read
Abdulmalik
Abdulmalik
AppSec Engineer

Do you even know the supply chain risk your Helm charts carry? I bet you don't.

Your app images get scanned in CI. Your GitOps repo still pins [email protected] from the public chart repo. Argo syncs it. Whatever containers that chart renders land in the cluster. Nobody looked. That is not a tooling gap. That is risk you are shipping on every sync.

I already wrote about SBOMs beyond generation and hardening GitHub Actions. This one is the missing middle: the ops repo. Argo Applications, Flux HelmReleases, Helm targetRevision pins, Kustomize bundles that yank GitHub release YAMLs. That tree is how third-party containers enter the cluster. If your scanners only run on application build pipelines, you are blind there.

So I open sourced the path I use: render what GitOps would sync, inventory the images, run Syft + Grype, gate on fixable High/Critical. That is supply chain security on the images the ops repo would deploy, not another app CI image scan.

The gap​

App image CI and chart-render supply chain are not the same control. Confusing them is how infra teams sleep well while the cluster stays soft.

App pipelines usually look like: build → SBOM → scan → maybe sign → push to ECR. Fine for your code.

Platform charts are different:

  • The "source" is an Argo Application (or Flux HelmRelease) with chart + repoURL + targetRevision
  • Images come from upstream defaults, or from helm.values overrides you forgot about
  • Tags are often mutable (:latest, :v1.2) with no digest
  • CVEs show up months after you pinned the chart, and the fix is almost never "edit the Deployment image by hand"

If you only scan the generic chart folder with helm lint, you never see the rendered image set for staging vs production. You never see the risk either.

What I wired​

Two repos, same job:

Modes: argo, flux, gitops (Argo + Flux), manifests, chart, terraform. inventory scopes what would deploy. scan is the gate (Syft → Grype, --only-fixed by default). recommend suggests chart / targetRevision bumps when a newer pin improves the Grype score.

Minimal Action workflow (pin the commit SHA):

name: ops supply chain

on:
  pull_request:
    paths:
      - "environments/**"

jobs:
  supply-chain:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

      - name: Scan Argo Application pins
        uses: saintmalik/helm-sca-action@cccf5a6f266ad6be7ca18655ad66b860e386e293 # v0.0.4
        with:
          mode: argo
          argo-apps: ./environments
          command: scan
          fail-on: high

Flux is the same shape with mode: flux and flux: ./clusters/prod. More examples live under the action repo.

CLI if you want it on a laptop or a non-Actions runner:

curl -sSfL \
  "https://github.com/saintmalik/helm-sca/releases/download/v0.0.2/helm-sca_linux_amd64.tar.gz" \
  | tar -xz -C /usr/local/bin helm-sca

helm-sca inventory --argo-apps ./apps --repo-root .
helm-sca scan --flux ./clusters --repo-root . --out-dir ./out --fail-on high
helm-sca recommend --argo-apps ./apps --repo-root . --out-dir ./out

Needs helm on PATH for chart/GitOps/Terraform paths. scan / recommend need Syft and Grype (the Action installs those). Kustomize sources need kubectl or kustomize.

What the pipeline does​

Against the ops repo (PRs or a scheduled job over staging / production trees):

  1. Render Argo Applications / Flux releases (and nested apps inside Kustomize) the same way the cluster would see them
  2. Inventory into a CycloneDX SBOM that maps app → chart/repo/revision → images
  3. Syft each unique image
  4. Grype with --only-fixed, fail on High/Critical when you set fail-on
  5. Warn if image refs lack @sha256:
  6. Recommend chart pin bumps when a newer chart version templates to fewer fixable High/Critical findings

Chart pin recommend is shipped: it resolves newer versions (helm search repo / helm show chart), re-templates current vs candidate, scores both image sets, and suggests a bump only when the Grype score improves. Private chart repos still need helm auth; version discovery is best-effort.

Render first, or you are lying to yourself​

The interesting part is not "run Grype." It is forcing the scan target to match sync.

For remote Helm Applications the helper does the boring thing Argo does:

helm template "$release" "$chart" \
  --repo "$repo_url" \
  --version "$target_revision" \
  -f <inline-values> \
  --skip-tests

OCI charts become oci://$repo/$chart. Local charts use the path + releaseName. Kustomize paths get kubectl kustomize, and nested Application docs inside the bundle get rendered too.

Local chart deps: if charts/*.tgz already exists, use them. Otherwise try helm dependency build and keep going with a warning if that fails. CI should not pretend vendored deps are optional forever, but a hard fail on every transient chart museum blip is also how people disable the whole job.

If nothing renders, the job exits. Empty scan = false green is worse than a red PR.

Two SBOMs, different jobs​

I generate:

  1. Deploy inventory (inventory.cdx.json): one component per app, with properties for source file, chart, repo, revision, path. Child components are the container images that came out of render.
  2. Per-image CycloneDX under the scan out-dir: Syft against the actual image (with registry login when the runner already has creds).

Grype prefers grype sbom:$file. If Syft could not pull a private image, fall back to scanning the image ref directly and warn. Private registry auth is the usual failure mode; understated CVE counts beat silent success.

The inventory is what makes the report useful. Finding is not "CVE in nginx." It is "this staging Application YAML, this chart pin, this image."

What the gate actually fails on​

Fixable High/Critical from Grype (--only-fixed). If upstream has not published a fixed version, a PR block does not magically create one. Tune fail-on (none / low / medium / high / critical) to how loud you want the check.

Mutable tags get a warning, not a fail (yet). Digest pinning is the right end state. Most ops repos do not get there in one PR.

Upload the scan out-dir as an artifact. Fail closed on actionable findings after your team actually reads the report.

Chart bumps, not image surgery​

This is the bit teams get wrong after the first Grype dump.

For external charts, bump targetRevision (or the Flux chart version / Kustomize GitHub release URL), not chase container tags inside rendered YAML. helm-sca recommend does that scoring:

  • Find remote Helm pins (Argo Application, Flux HelmRelease, Terraform helm_release, …)
  • Resolve newer versions via helm search repo / helm show chart
  • Re-template current pin and candidate pin(s)
  • Score fixable High/Critical on both image sets (Syft → Grype --only-fixed)
  • Say bump only when the newer chart actually improves the count
  • Call out values that override tag / repository / registry, because a chart bump alone will not move those images
helm-sca recommend --argo-apps ./environments --repo-root . --out-dir ./out

Artifacts: upgrade-recommendations.md + .json. Limits: private chart repos need helm auth; version discovery is best-effort.

Also: your own base images in ECR are not fixed by bumping Bitnami. Track those internally.

Adjacent layer: signing your own images​

App build pipelines still sign what you push (Notation + AWS Signer in my setup). That is a different control: authenticity of internal images.

It does not tell you that the ingress-nginx chart you pinned six months ago still renders a controller image with a fixable Critical. Keep both. Do not confuse chart provenance theater with "what is about to run."

I am not verifying Helm chart signatures in this path today. If your threat model needs that, add it. This path answers a blunter question: given the pins in git, what CVEs ship on sync?

Tradeoffs​

Pros

  • Scans the same rendered surface GitOps syncs
  • Maps CVEs back to Application / HelmRelease files humans can edit
  • Pushes remediation toward chart/release pins

Cons / sharp edges

  • Needs network to chart repos and registries during CI
  • Private images without auth under-report
  • Chart "latest" lookup heuristics are imperfect (best-effort helm search / helm show)
  • Mutable tags are warned, not blocked
  • Render failures are skipped with warnings; a broken Application can disappear from the scan until you watch the logs
  • Private charts need helm repo/registry auth or recommend cannot resolve versions

Also: this is CI-time on the ops repo. Runtime drift (someone helm upgrade outside GitOps) is out of scope. That is a different detector.

Conclusion​

GitOps did not remove your Helm supply chain. It moved it into YAML that looks too boring to scan.

Render it. Inventory it. Grype the images. Gate the ops repo. Then argue about digests, chart signatures, and auto pin-bumps once the boring path is green.

Till next time, Peace be on you 🙏🏽

References​


Comments