Engin Diri
_CLOUD

_CLOUD

Drop ballast... How I reduced the size of minectl ๐Ÿ—บ

Drop ballast... How I reduced the size of minectl ๐Ÿ—บ

with golang, upx and goreleaser

Engin Diri's photo
Engin Diri

Published on Aug 10, 2021

4 min read

Looking at the current state of minectl ๐Ÿ—บ, I think it reached now a good level in terms of features and functionalities. So it is time for me, to open the next big chapter:

Getting minectl ๐Ÿ—บ CI ready.

image.png

Before I could start with this, I wanted to be really sure that I did everything during the development and creation phase of the actual minectl ๐Ÿ—บ binary.

So the first thing, what comes in to my mind was to verify the size of the binary.

To be honest, I don't want that the filesize of minectl ๐Ÿ—บ becomes an issue, and users have to wait longer then necessary.

Having to wait to download an unreasonable sized binary (+container) is not really a good thing.

So let's have a look into the raw binary size of minectl ๐Ÿ—บ:

โžœ  minectl git:(main) โœ— go build .                              
โžœ  minectl git:(main) โœ— ls -lah minectl 
-rwxr-xr-x  1 dirien  staff    58M Aug 10 22:50 minectl
โžœ  minectl git:(main) โœ—

image.png

A quick look on the entropy graph (made with binwalk) reveals that we have potential to compress the binary further down.

graph1.png

Short explanation of the entropy graph:

An entropy graph (to evaluate the amount of disorder) can be useful to detect the parts of the file that get close to random data.

It will allow to detect the parts that have been encrypted/compressed and the parts that appear to be left untouched.

Okay, time to drop some ballast.

Golang home remedies

Let us strip the binary from the debugging information. To do this we can use the -s and -w linker flags.

โžœ  minectl git:(main) โœ— go build -ldflags="-s -w" .
โžœ  minectl git:(main) โœ— ls -lah minectl 
-rwxr-xr-x  1 dirien  staff    48M Aug 10 23:33 minectl
โžœ  minectl git:(main) โœ—

With this little action we nearly reduce the size to 82% of the initial size. Quick look in the new entropy graph:

minectl.png

We can see that we lost a big chunk of padding.

We can to better than this! Open the stage for UPX

UPX - The Ultimate Packer for eXecutables

upx is an advanced executable file compressor. upx will typically reduce the file size of programs by around 50%-70%, thus reducing disk space, network load times, download times and other distribution and storage costs.

So let us see what upx can do for us. Running upx, in vanilla mode, gives us following results:

โžœ  minectl git:(main) โœ— upx minectl
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2020
UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  50085008 ->  12988432   25.93%   macho/amd64   minectl                       

Packed 1 file.
โžœ  minectl git:(main) โœ— ls -lah minectl
-rwxr-xr-x  1 dirien  staff    12M Aug 10 23:33 minectl

So with the -s and -w flag and upx, we reduced the size to 20% of the initial binary. Reducing the overall size to nice 12M.

The new entropy graph looks like this:

graph-upx.png

Can we squeeze more out of our minectl ๐Ÿ—บ binary? Let us try to run upx in brute mode. This takes a little more time, but hey if it helps I can wait on my production build.

image.png

โžœ  minectl git:(main) โœ— upx --brute minectl
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2020
UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  50085008 ->   8732688   17.44%   macho/amd64   minectl                       

Packed 1 file.
โžœ  minectl git:(main) โœ— ls -lah minectl
-rwxr-xr-x  1 dirien  staff   8.3M Aug 10 23:50 minectl

8.3M is the binary size, which is whopping 13% of the initial binary.

Our entropy graph is now very boring:

graph-brute.png

The downside: decompression is not free, and we will get some overhead when starting the minectl ๐Ÿ—บ. We're talking here in ms. I think that this is in a totally acceptable area.

Connect the dots

GoReleaser

24697112.png

As I am using goreleaser for building my binaries and container image, I can easily add the upx call into the .goreleaser.yml:

builds:
  -
...
    hooks:
      post:
       - upx --brute "{{ .Path }}"

Container Build

To avoid bloating the container up, now we spent some time to reduce it. I am going to use an alpine base image for my container.

FROM alpine:3.14.1
COPY minectl \
    /usr/bin/minectl
ENTRYPOINT ["/usr/bin/`minectl`"]

Thats it. With a relative low effort, we achieved our goal to reduce the size of minectl ๐Ÿ—บ.

image.png

Credits

ย 
Share this