profile

Ivan Velichko

Container Tools, Tips, and Tricks - Issue #1

Published over 1 year ago • 2 min read

Using Docker For Building... Code

A pilot issue of my new mid-of-the-month newsletter.

Typically, Docker is used for building container images and/or running containerized programs. However, it can easily be (ab)used as a generic build tool. Let me show you a few examples how it can come in handy.

Hacking stuff quickly

These days I use Rust only occasionally. In particular, it means there is often no configured development environment around. So, whenever there is a new PR in one of my Rust projects, here is what I do:

$ git clone https://github.com/iximiuz/reapme.git
$ cd reapme

$ docker run -v $(pwd):/app -w /app rust:1 cargo run --bin subreaper

The above docker run command starts a temporary build environment where I can validate the changes quickly. Alternatively, if the build artifacts need to be used outside of the builder container, the command can be adjusted as follows:

$ docker run -v $(pwd):/app -w /app rust:1 cargo build --release

$ file target/release/subreaper
target/release/subreaper: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV)...

Obviously, this trick works not only for Rust. Java and Go are two other great candidates, and the technique can be extended beyond compiling tiny programs. For instance, you can minify a bunch of CSS files without spoiling your system with any Node.js tools. As someone who's been advocating for disposable and isolated development environments, I find it very appealing.

The downside, of course, is the time needed for a new container to pull in and compile all the dependencies. So, if your use case is slightly less ephemeral, you should probably think of setting up a proper dev environment.

Leveraging BuildKit superpowers

Since version 18.09 (released in ~2019), Docker relies on BuildKit for building container images. However, BuildKit isn't limited to building images. It defines itself as a toolkit for converting source code to build [arbitrary] artifacts in an efficient, expressive, and repeatable manner.

twitter profile avatar
Ivan Velichko
Twitter Logo
@iximiuz
October 15th 2022
2
Retweets
15
Likes

With the docker buildx build command, changing the type of the build artifact becomes as simple as providing the --output flag:

$ docker buildx build --output type=local,dest=path/to/artifacts

# Or a shorter form:
$ docker buildx build -o path/to/artifacts

By describing your build procedure in a Dockerfile, you immediately start benefiting from efficient (and potentially remote) caching, isolated build environments, out-of-the-box cross-platform builds, and more. And the build results don't need to be container images! It'll also work for producing normal local files.

Read more about the BuildKit superpowers in this highly practical article by Batuhan Apaydın.

Harnessing Bake - Docker-aware Make

It's a common practice to invoke docker build commands from a Makefile. GNU make can run independent recipes in parallel, but unlike BuildKit, it's not a container-native build system. If the build targets were described in a format that BuildKit understands, it could handle multiple concurrent build requests much more efficiently than make by de-duplicating the build steps, using more advanced caching, remote build instances, etc, etc.

And that's how the new docker buildx bake command was born.

You can think of bake as an HCL-based container-aware make:

# docker-bake.hcl
group "default" {
  targets = ["db", "webapp-dev"]
}

target "webapp-dev" {
  dockerfile = "Dockerfile.webapp"
  tags = ["docker.io/username/webapp:latest"]
}

target "webapp-release" {
  inherits = ["webapp-dev"]
  platforms = ["linux/amd64", "linux/arm64"]
}

target "db" {
  dockerfile = "Dockerfile.db"
  tags = ["docker.io/username/db"]
}

Combining bake with the --output trick gives you an efficient, highly concurrent, and isolated (from the host system) build tool. Go give it a try!

Random (but slightly related) tweet

Look ma', no Dockerfile!

twitter profile avatar
Ivan Velichko
Twitter Logo
@iximiuz
October 16th 2022
21
Retweets
142
Likes

Have a productive week ahead!

Ivan

Ivan Velichko

Software Engineer at day. Tech Storyteller at night. Helping people master Containers.

Read more from Ivan Velichko

Hello friends! Ivan's here - with a well overdue February roundup of all things Linux, Containers, Kubernetes, and Server-Side craft 🧙 What I was working on A lot of stuff on the dev side - not so much on the content side. But things are soon to reverse 🤞 Announcing labCTL - the long-awaited iximiuz Labs CLI A dozen people have asked me over the past year-ish if there'll be access to the playgrounds from the local terminal and not only from the browser. And while I myself wanted this feature...

about 1 month ago • 7 min read

Hello there! 👋 Debugging containerized applications is... challenging. Debugging apps that use slim variants of container images is double challenging. And debugging slim containers in hardened production environments is often close to impossible. Before jumping to the DevOps problems that I prepared for you this week, let's review a few tricks that can be used to troubleshoot containers. If the container has a shell inside, running commands in it with docker exec (or kubectl exec) is...

about 2 months ago • 1 min read

Hey hey! Are you ready for your next DevOps challenge? Last week, we all witnessed yet another terrifying cyber-security event, and this time, it was a direct hit - researchers from Snyk discovered a way to break out of containers! 🤯 The vulnerability was found in the fundamental component of the containerization ecosystem - the most popular implementation of the (low-level) OCI container runtime - runc. Notice how, on the diagram above, most high-level container runtimes actually rely on the...

2 months ago • 1 min read
Share this post