Drop ballast... How I reduced the size of minectl 🗺

Looking at the current state of minectl 🗺, I think it reached now a good level in terms of features and functionalities. So it is time for me, to open the next big chapter:

Getting minectl 🗺 CI ready.

Before I could start with this, I wanted to be really sure that I did everything during the development and creation phase of the actual minectl 🗺 binary.

So the first thing, what comes in to my mind was to verify the size of the binary.

To be honest, I don't want that the filesize of minectl 🗺 becomes an issue, and users have to wait longer then necessary.

Having to wait to download an unreasonable sized binary (+container) is not really a good thing.

So let's have a look into the raw binary size of minectl 🗺:

➜  minectl git:(main) ✗ go build .                              
➜  minectl git:(main) ✗ ls -lah minectl 
-rwxr-xr-x  1 dirien  staff    58M Aug 10 22:50 minectl
➜  minectl git:(main) ✗

A quick look on the entropy graph (made with binwalk) reveals that we have potential to compress the binary further down.

Short explanation of the entropy graph:

An entropy graph (to evaluate the amount of disorder) can be useful to detect the parts of the file that get close to random data.

It will allow to detect the parts that have been encrypted/compressed and the parts that appear to be left untouched.

Okay, time to drop some ballast.

Golang home remedies

Let us strip the binary from the debugging information. To do this we can use the -s and -w linker flags.

➜  minectl git:(main) ✗ go build -ldflags="-s -w" .
➜  minectl git:(main) ✗ ls -lah minectl 
-rwxr-xr-x  1 dirien  staff    48M Aug 10 23:33 minectl
➜  minectl git:(main) ✗

With this little action we nearly reduce the size to 82% of the initial size. Quick look in the new entropy graph:

We can see that we lost a big chunk of padding.

We can to better than this! Open the stage for UPX

UPX - The Ultimate Packer for eXecutables

upx is an advanced executable file compressor. upx will typically reduce the file size of programs by around 50%-70%, thus reducing disk space, network load times, download times and other distribution and storage costs.

So let us see what upx can do for us. Running upx, in vanilla mode, gives us following results:

➜  minectl git:(main) ✗ upx minectl
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2020
UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  50085008 ->  12988432   25.93%   macho/amd64   minectl                       

Packed 1 file.
➜  minectl git:(main) ✗ ls -lah minectl
-rwxr-xr-x  1 dirien  staff    12M Aug 10 23:33 minectl

So with the -s and -w flag and upx, we reduced the size to 20% of the initial binary. Reducing the overall size to nice 12M.

The new entropy graph looks like this:

Can we squeeze more out of our minectl 🗺 binary? Let us try to run upx in brute mode. This takes a little more time, but hey if it helps I can wait on my production build.

➜  minectl git:(main) ✗ upx --brute minectl
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2020
UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  50085008 ->   8732688   17.44%   macho/amd64   minectl                       

Packed 1 file.
➜  minectl git:(main) ✗ ls -lah minectl
-rwxr-xr-x  1 dirien  staff   8.3M Aug 10 23:50 minectl

8.3M is the binary size, which is whopping 13% of the initial binary.

Our entropy graph is now very boring:

The downside: decompression is not free, and we will get some overhead when starting the minectl 🗺. We're talking here in ms. I think that this is in a totally acceptable area.

Connect the dots

GoReleaser

As I am using goreleaser for building my binaries and container image, I can easily add the upx call into the .goreleaser.yml:

builds:
  -
...
    hooks:
      post:
       - upx --brute "{{ .Path }}"

Container Build

To avoid bloating the container up, now we spent some time to reduce it. I am going to use an alpine base image for my container.

FROM alpine:3.14.1
COPY minectl \
    /usr/bin/minectl
ENTRYPOINT ["/usr/bin/`minectl`"]

Thats it. With a relative low effort, we achieved our goal to reduce the size of minectl 🗺.

_CLOUD