In the last story, we talked about, how we successfully reduced the size of our minectl 🗺 cli container. The idea behind this all is to get minectl 🗺 ready to be used in a pipeline.

So what else can we do?

Was there not something with Docker Content Trust/Notary. Yeah, but that never really went mainstream in version v1. Don't know about the v2, and the progress there.

What else could we use? Of course cosign!. You probably read about it on Twitter, here a recent tweet from Ahmet Alp Balkan

Or all tweets, from the mastermind himself Dan Lorenc

The cosign project is part of the sigstore initiative. They have really cool ideas about the whole open source security topic. For now, we stick with the image signing process.

That means that minectl 🗺 has to fit nicely into a secure toolchain. So anyone downloading the minectl 🗺, can be sure it is build from me or a system with access to the private key.

So what do we need to get everything in place:

Ze tools:

cosign

There are several ways to download the sigstore binary, via brew or directly from the release page.

arkade

This time I want to download it with arkade, a tool Alex Ellis wrote and which provides a portable marketplace for downloading your favourite devops CLIs and installing helm charts, with a single command.

arkade get cosign
Downloading: cosign
Tool written to: /Users/dirien/.arkade/bin/cosign

# Add (cosign) to your PATH variable
export PATH=$PATH:$HOME/.arkade/bin/

# Test the binary:
/Users/dirien/.arkade/bin/cosign

# Or install with:
sudo mv /Users/dirien/.arkade/bin/cosign /usr/local/bin/

And there its, ready to use for minectl 🗺.

cosign version
GitVersion:    v1.0.0
GitCommit:     33973d078170f586cf27f2cc464844b3f1fa1abb
GitTreeState:  clean
BuildDate:     '2021-07-28T14:38:03Z'
GoVersion:     go1.16.6
Compiler:      gc
Platform:      darwin/amd64

Lets step back and, verify cosign with cosign. Just for the lolz. They have the public key in their git repository and also the signature of the binary.

wget https://raw.githubusercontent.com/sigstore/cosign/main/release/release-cosign.pub

wget https://github.com/sigstore/cosign/releases/download/v1.0.0/cosign-darwin-amd64.sig

cosign verify-blob -key release-cosign.pub -signature cosign-darwin-amd64.sig /usr/local/bin/cosign
Verified OK

Everything is fine.

Now, back to minectl 🗺: The actual steps, are quite easy and straight forward:

Generate a keypair

Choose a long password for the private key.

➜  minectl git:(main) ✗ cosign generate-key-pair
Enter password for private key: 
Enter again: 
Private key written to cosign.key
Public key written to cosign.pub
➜  minectl git:(main) ✗

I am going to commit the public key into my repo. So everybody can use this to verify the signature of the binaries and container images.

goreleaser

That's one part. As I use goreleaser for my releases, we will add the creation of the signature file for the binaries into the post hook of the build section in the .goreleaser.yaml

...
 hooks:
      post:
        - upx "{{ .Path }}"
        - sh -c "cosign sign-blob -key $COSIGN_KEY_LOCATION {{ .Path }} > dist/{{ .ProjectName }}_{{ .Version }}_{{ .Target }}.sig"
...

To upload the signature files to the current release, I need to add them to the releasetag too.

...
release:
  extra_files:
    - glob: dist/*.sig

To sign the container images, whe need to extend our Github pipelines, in several places. First we need to install cosign.

Cosign can easily be installed in your GitHub actions using sigstore/cosign-installer.

Thanks to the great work of Carlos Panato

Next step is to add the COSIGN_KEY and cosign key password COSIGN_PASSWORD to Github as a repository secret

That way, we can easily inject the values into our pipeline.

As we need to use cosign in goreleaser and in a specific pipeline step, we create the private key with the content of the variable COSIGN_KEY. The expected location is defined in the COSIGN_KEY_LOCATION variable.

Last action in the pipeline is more of a workaround, while goreleaser has no post hook in the docker build.

So we need to get the version from the git tag command and build the image name by hand. Luckily, we only support linux container.

I will create an issue for this in the goreleasergithub repository.

Github Actions

....
      - uses: sigstore/cosign-installer@main
        with:
          cosign-release: 'v1.0.0'
      - name: install cosign private key
        run: 'echo "$COSIGN_KEY" > $COSIGN_KEY_LOCATION'
        shell: bash
        env:
          COSIGN_KEY: ${{secrets.COSIGN_KEY}}
      - name: Run GoReleaser
        uses: goreleaser/goreleaser-action@v2
        with:
          version: latest
          args: release --rm-dist
        env:
          GITHUB_TOKEN: ${{ secrets.GH_PAT }}
          COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
      - name: set version as env
        run: |
          tag=$(git describe --tags --abbrev=0 | tr --delete v)
          echo "version=$tag" >> $GITHUB_ENV
      - name: Sign the image
        run: |
          echo -n "${{secrets.COSIGN_PASSWORD}}" | cosign sign -key $COSIGN_KEY_LOCATION ghcr.io/dirien/minectl:${{ env.version }}-arm64
          echo -n "${{secrets.COSIGN_PASSWORD}}" | cosign sign -key $COSIGN_KEY_LOCATION ghcr.io/dirien/minectl:${{ env.version }}-amd64

If everything works well, you should see following output in your pipeline logs:

Same goes for the automatic signing of the image:

Let us check the image with the public key:

➜  minectl git:(sign) ✗ cosign verify -key cosign.pub ghcr.io/dirien/minectl:0.8.0-amd64

Verification for ghcr.io/dirien/minectl:0.8.0-amd64 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
  - Any certificates were verified against the Fulcio roots.
{"critical":{"identity":{"docker-reference":"ghcr.io/dirien/minectl"},"image":{"docker-manifest-digest":"sha256:1d72531b7c4cc1df20da8e05648a6662b18219dc9a016fef7b446f5bbe7d4e0c"},"type":"cosign container image signature"},"optional":null}

That's it, now we can continue with our story to make minectl 🗺 ready for pipeline usage.

If you want to see everything in action -> Head over to the minectl 🗺 repo github.com/dirien/minectl

_CLOUD