Keyless Signing of Container Images using GitHub Actions

August 15, 2023 · 8 min read

AppSec Engineer

In my past article about signing container images, got some comments which led me to digging about keyless signing of container images.

You will be walked through the process of keyless signing of container images using GitHub Actions.

Prerequisite

GitHub Account
Knowledge about GitHub Action

So Lets jump into it;

👉 How does the keyless signing works?

keyless signing works by verifying the signer's identity by using identity providers like Google, Email and GitHub and it puts the signers identity into the artifact signing certificate.

When the signing is done, the signing certificate gets thrown away after 10 minutes, and the only metadata from the whole act is the public key stored inside the Rekor tlog.

Cosign: a tool that signs software artifacts, this brings trust and provenance to the software and helps prevent tampering.
And fulcio: on the other hand is a free code signing certificate authority based on an OpenID Connect Email address, Fulcio signs X.509 certificates valid for 10 minutes.
Rekor: also known as transparency log that holds metadata generated within a software project’s supply chain signing and allows other users of the software to query the logs to see if the signature is valid and signed by the authorized software owners.

You can host your personal rekor instance instead of using the public one if your container images is private and you dont want to have it uploaded to public sigstore rekor instance.

👉 Keyless Signing of Container image with GitHub Actions

You need to create a GitHub Action YAML file named keyless.yaml in the your github repository in the folder path .github/workflows and write the following syntaxs in it.

You have to start with the permmision section, the workflow will be needing the following permissions.

.github/workflows/keyless.yaml
name: keyless signing container images
on: push

jobs:
   build-keyless-signing-container-image::
    runs-on: ubuntu-latest
    permissions:
        contents: read
        id-token: write

The id-token: write enables the GitHub Actions OIDC tokens for your workflow, so that way Fulcio will be able to do it's job without needing you to hit the auth verification url manually to select the OIDC method you want to use.

You still have to be care and make sure the permission isnt available on the pull request based run for that action, you can read more from the github security hardening

Now, you write the actions to build your container images from your Dockerfile, the Dockerfile used in this guide is below, you can use it too for practical sake.

Dockerfile
FROM
FROM alpine
RUN apk update
RUN apk add git

So here is the actions to build the container image from the dockerfile and push the container image to a docker registry

.github/workflows/keyless.yaml
    steps:
      - name: "checkout"
        uses: actions/checkout@v3

      - name: Generate uuid from shortened commit SHA
        id: uuid
        run: echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"

      - name: Build and push
        env:
          IMAGE_TAG: signed-test-${{ steps.uuid.outputs.sha_short }}
        run: |
          docker build -t ttl.sh/${IMAGE_TAG}:1h .
          docker push ttl.sh/${IMAGE_TAG}:1h

in the above workflow, i am using the ttl.sh container repositories which is an Anonymous & ephemeral Docker image registry, you can always use any container repository you want.

Next step in the workflow is getting the image digest of the container image built in the above step using the below actions in your workflow.

.github/workflows/keyless.yaml
      - name: Get image digest
        env:
          IMAGE_TAG: signed-test-${{ steps.uuid.outputs.sha_short }}
        id: digest
        run: |
          echo "image_sha=$(docker inspect --format='{{index .RepoDigests 0}}' ttl.sh/${IMAGE_TAG}:1h)" >> $GITHUB_OUTPUT

Next step in your workflow is installing Cosign, add the following YAML config in your workflow

.github/workflows/keyless.yaml
      - name: Install cosign
        uses: sigstore/cosign-[email protected]
        with:
          cosign-release: 'v2.1.1'

The next step is the most important in our workflow, signing the container image we've built and pushed with the following actions.

.github/workflows/keyless.yaml
      - name: Keyless signing of image
        run: |
          cosign sign --yes --rekor-url "https://rekor.sigstore.dev/" ${{ steps.digest.outputs.image_sha }}

The ${{ steps.digest.outputs.image_sha }} is the output of the step where you ran the actions to grab your container image digest, you will also notice i did specify the --rekor-url flag, this is needed for when you have your own private rekor instance.

So its a must to set the rekor-url and fulcio-url flag if you have hosted your own private rekor and fulcio server.

If everything goes well, your signing workflow tlog output should look just like this, with information about the rekor tlog indexing number, the container image, SHA value of our image tag.

👉 Verifying and enforcing signed container images policies

Verifying

Now its time to verify and enforce that only verified image are used in our environments, to verify your just built container image.

Run the following cosign verify command on your local system.

cosign verify  --rekor-url "https://rekor.sigstore.dev/" ttl.sh/signed-test-89b280e@sha256:9cbba3d51f93e5ccaea502009a72bfb88985cc9c179982574f012394f45edd4d --certificate-identity "https://github.com/your-github-username/your-github-repo.github/workflows/keyless.yaml@refs/heads/main" --certificate-oidc-issuer "https://token.actions.githubusercontent.com" | jq .

Replace ttl.sh/signed-test-89b280e@sha256:9cbba3d51f93e5ccaea502009a72bfb88985cc9c179982574f012394f45edd4d with your own container image URL, likewise the your-github-username with your github username and your-github-repo with your own repo name.

Or better still run it in your github actions workflow.

.github/workflows/keyless.yaml
      - name: Verify the image signing
        run: |
          cosign verify  --rekor-url "https://rekor.sigstore.dev/" ${{ steps.digest.outputs.image_sha }} --certificate-identity "https://github.com/your-github-username/your-github-repo/.github/workflows/keyless.yaml@refs/heads/main" --certificate-oidc-issuer "https://token.actions.githubusercontent.com" | jq .

And your output should look something like this.

Output
Verification for ttl.sh/signed-test-89b280e@sha256:9cbba3d51f93e5ccaea502009a72bfb88985cc9c179982574f012394f45edd4d --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates
[
  {
    "critical": {
      "identity": {
        "docker-reference": "ttl.sh/signed-test-89b280e"
      },
      "image": {
        "docker-manifest-digest": "sha256:9cbba3d51f93e5ccaea502009a72bfb88985cc9c179982574f012394f45edd4d"
      },
      "type": "cosign container image signature"
    },
    "optional": {
      "Bundle": {
        "SignedEntryTimestamp": "MEUCIQCU2iiGHGpGn4F5Z052N6E3tGDe006msAaFfWNYC0YvnQIgePYnhb/5wIL+0ZtKgAaFTS3lzSmnA9eEWxHENc7Hgy4=",
        "Payload": {
          "body": "xxxxxxx",
          "integratedTime": 1692204473,
          "logIndex": 31528445,
          "logID": "c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d"
        }
      },
      "Issuer": "https://token.actions.githubusercontent.com",
      "Subject": "https://github.com/saintmalik/sign-container-images/.github/workflows/keyless.yaml@refs/heads/main",
      "githubWorkflowName": "keyless signing container images",
      "githubWorkflowRef": "refs/heads/main",
      "githubWorkflowRepository": "saintmalik/sign-container-images",
      "githubWorkflowSha": "89b280e04139b13a002712574c76d09ffdba3d67",
      "githubWorkflowTrigger": "push"
    }
  }
]

And your final workflow should look like this

.github/workflows/keyless.yaml
name: keyless signing container images
on: push

jobs:
 build-sign-container-image:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - name: "checkout"
        uses: actions/checkout@v3

      - name: Generate uuid from shortened commit SHA
        id: uuid
        run: echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"

      - name: Build and push
        env:
          IMAGE_TAG: signed-test-${{ steps.uuid.outputs.sha_short }}
        run: |
          docker build -t ttl.sh/${IMAGE_TAG}:1h .
          docker push ttl.sh/${IMAGE_TAG}:1h

      - name: Get image digest
        env:
          IMAGE_TAG: signed-test-${{ steps.uuid.outputs.sha_short }}
        id: digest
        run: |
          echo "image_sha=$(docker inspect --format='{{index .RepoDigests 0}}' ttl.sh/${IMAGE_TAG}:1h)" >> $GITHUB_OUTPUT

      - name: Install cosign
        uses: sigstore/cosign-[email protected]
        with:
          cosign-release: 'v2.1.1'

      - name: Keyless signing of image
        run: |
          echo ${{ steps.digest.outputs.image_sha }}
          cosign sign --yes --rekor-url "https://rekor.sigstore.dev/" ${{ steps.digest.outputs.image_sha }}

      - name: Verify the image signing
        run: |
          cosign verify  --rekor-url "https://rekor.sigstore.dev/" ${{ steps.digest.outputs.image_sha }} --certificate-identity "https://github.com/saintmalik/sign-container-images/.github/workflows/keyless.yaml@refs/heads/main" --certificate-oidc-issuer "https://token.actions.githubusercontent.com" | jq .

Enforce Policy

Enforcing the policy of using only signed container image can be done at any level, you can enforce it at the gitops level using OPA, making sure only signed image makes it through to your gitops repo or even using Kubernetes Policy Controller for the Kubernetes users.

I prefer kyverno though, you can simply enforce a cluster wide policy to make sure in whatever namespace a deployement is, if the image matches the one you are enforcing the policy for.

It must check if it's signed based on the attestors you've given it, else don't allow the deployment make it through, here is a sample.

signed-image-deployment.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: check-signed-image
spec:
  validationFailureAction: enforce
  background: false
  webhookTimeoutSeconds: 30
  failurePolicy: Fail
  rules:
    - name: check-image
      match:
        any:
        - resources:
            kinds:
              - Pod
      verifyImages:
      - imageReferences:
        - "ttl.sh/*:*"
        verifyDigest: false
        required: false
        mutateDigest: false
        attestors:
        - entries:
          - keyless:
              # verifies issuer and subject are correct
              issuer: https://token.actions.githubusercontent.com
              subject: https://github.com/saintmalik/sign-container-images/.github/workflows/keyless.yaml@refs/heads/main
              rekor:
                url: https://rekor.sigstore.dev

signed-image-deployment.yaml
      resources:
        limits:
          memory: 384Mi
        requests:
          cpu: 100m
          memory: 128Mi
      volumeMounts:
        - name: public-keys
          mountPath: /public-keys
        - name: sigstore
          mountPath: /.sigstore

  volumes:
    - name: sigstore
      emptyDir: {}
    - name: public-keys
      configMap:
        name: public-keys

Well, that's it folks! I hope you find this piece insightful and helpful.

Till next time ✌️

Prerequisite​

👉 How does the keyless signing works?​

👉 Keyless Signing of Container image with GitHub Actions​

👉 Verifying and enforcing signed container images policies​

Verifying​

Enforce Policy​

References​

Comments