Container Tools, Tips, and Tricks - Issue #1

published12 months ago
2 min read

Using Docker For Building... Code

A pilot issue of my new mid-of-the-month newsletter.

Typically, Docker is used for building container images and/or running containerized programs. However, it can easily be (ab)used as a generic build tool. Let me show you a few examples how it can come in handy.

Hacking stuff quickly

These days I use Rust only occasionally. In particular, it means there is often no configured development environment around. So, whenever there is a new PR in one of my Rust projects, here is what I do:

$ git clone https://github.com/iximiuz/reapme.git
$ cd reapme

$ docker run -v $(pwd):/app -w /app rust:1 cargo run --bin subreaper

The above docker run command starts a temporary build environment where I can validate the changes quickly. Alternatively, if the build artifacts need to be used outside of the builder container, the command can be adjusted as follows:

$ docker run -v $(pwd):/app -w /app rust:1 cargo build --release

$ file target/release/subreaper
target/release/subreaper: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV)...

Obviously, this trick works not only for Rust. Java and Go are two other great candidates, and the technique can be extended beyond compiling tiny programs. For instance, you can minify a bunch of CSS files without spoiling your system with any Node.js tools. As someone who's been advocating for disposable and isolated development environments, I find it very appealing.

The downside, of course, is the time needed for a new container to pull in and compile all the dependencies. So, if your use case is slightly less ephemeral, you should probably think of setting up a proper dev environment.

Leveraging BuildKit superpowers

Since version 18.09 (released in ~2019), Docker relies on BuildKit for building container images. However, BuildKit isn't limited to building images. It defines itself as a toolkit for converting source code to build [arbitrary] artifacts in an efficient, expressive, and repeatable manner.

twitter profile avatar
Ivan Velichko
Twitter Logo
@iximiuz
October 15th 2022
2
Retweets
15
Likes

With the docker buildx build command, changing the type of the build artifact becomes as simple as providing the --output flag:

$ docker buildx build --output type=local,dest=path/to/artifacts

# Or a shorter form:
$ docker buildx build -o path/to/artifacts

By describing your build procedure in a Dockerfile, you immediately start benefiting from efficient (and potentially remote) caching, isolated build environments, out-of-the-box cross-platform builds, and more. And the build results don't need to be container images! It'll also work for producing normal local files.

Read more about the BuildKit superpowers in this highly practical article by Batuhan Apaydın.

Harnessing Bake - Docker-aware Make

It's a common practice to invoke docker build commands from a Makefile. GNU make can run independent recipes in parallel, but unlike BuildKit, it's not a container-native build system. If the build targets were described in a format that BuildKit understands, it could handle multiple concurrent build requests much more efficiently than make by de-duplicating the build steps, using more advanced caching, remote build instances, etc, etc.

And that's how the new docker buildx bake command was born.

You can think of bake as an HCL-based container-aware make:

# docker-bake.hcl
group "default" {
  targets = ["db", "webapp-dev"]
}

target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
}

target "webapp-release" {
  inherits = ["webapp-dev"]
  platforms = ["linux/amd64", "linux/arm64"]
}

target "db" {
  dockerfile = "Dockerfile.db"
  tags = ["docker.io/username/db"]
}

Combining bake with the --output trick gives you an efficient, highly concurrent, and isolated (from the host system) build tool. Go give it a try!

Random (but slightly related) tweet

Look ma', no Dockerfile!

twitter profile avatar
Ivan Velichko
Twitter Logo
@iximiuz
October 16th 2022
21
Retweets
142
Likes

Have a productive week ahead!

Ivan

Ivan Velichko

Software Engineer at day. Tech Storyteller at night. Helping people master Containers.

Read more from Ivan Velichko