Hi friends!

It's Ivan again, with my traditional roundup of all things around iximiuz.com.

This month was mostly about container images and container runtimes. Here is what I've got for you:

• Some random container thoughts and explanatory drawings
• A bunch of handy tricks I wish I had learned sooner on my Cloud Native journey
• A couple of streams on containerization tech (DockerSlim and containerd)
• A new GitHub repository with yet another awesome list of resources
• One big and one small(er) blog posts based on my recent research
• And, of course, a whole lot of great reads, podcasts, and videos 🔥

Let's get started!

SPONSORED Packages and software supply chains get compromised more and more often. So, everyone these days should be a security expert. I already confined every my project to a separate VM, but how can I know if there is some malicious activity happening inside? Learn how to use the Linux audit system in this practical post by Teleport (no marketing fluff).

​

Container Runtime Visualised

While preparing for one of the workshops, I came up with the following visual explanation of the low-level container runtime:

If this topic sounds interesting, you can find a few useful links in this Twitter thread.

​

Random Evening Thoughts

Might be not fully accurate, but this explanation could help get through a popular misconception:

Ivan Velichko @iximiuz Containers vs Virtual Machines 🤔 - When you start a container, you get a virtualized OS. - When you start a VM, you get virtualized hardware. As simple as just that. But a VM w/o an OS isn't useful, so de facto, the term VM became a synonym for "VM+OS". Hence, the confusion. September 24th 2022 34 Retweets 294 Likes

​

Docker Tips & Tricks

To improve your container game 😎

Dealing With Containers Faster

Since (a very recent) discovery, this trick has saved me hours! Turns out, you can exec, stop, kill, remove, etc docker containers using just a short prefix of their IDs! Instead of typing (or rather copy-pasting):

docker rm b5758e341c2a

...you can just go with:

docker rm b5

Credit goes to Michael Irwin!

​

Store Build Results In Local Folders

Another Docker trick that I learned on Twitter (thanks to Chris Guest). Turns out, the buildx plugin supports this for quite a while:

docker buildx build -o <some-dir> .

This is pretty powerful since it allows you to use Docker as a build system. You can find more details here (bake is the keyword to look for).

Non-Docker Tips & Tricks

Not so much about containers, but I must share them:

• ​GitHub code search now supports the exact match - you just need to make your search string quoted (using '). I rely on the global GitHub code search a lot to figure out how to use a package or a library. Often it's more efficient than reading the documentation.
• ​You can use sourcegraph.com/search to search all GitHub and GitLab public repos - I didn't know that, and GitHub's own code search was good enough for me, but now I may reconsider. And it's very generous to provide this functionality for free, actually!

​

What I Was... Talking About

This is a new (and quite challenging) category for me, but I'm glad I gave it a try:

• ​Hands-on Introduction to DockerSlim | Rawkode Live - Learn how to create extremely small container images using DockerSlim (and what magic is hidden behind the tool). As always on Rawkode Academy, it was a highly practical session.
• ​containerd - a secret hero of the Cloud Native world - My first ever workshop where you can learn about the most widespread container runtime and how you can use it to dig deeper into containers, troubleshoot Kubernetes issues, or even as a building block for your own software.

​

What I Was Writing

On the blog, Twitter, ...and GitHub!

Awesome Container Tinkering

A yet another awesome list of resources? Nope, it's a list of awesome resources!

I started this GitHub project to collect links to (and thoughts about):

• Lower-level building blocks (runtimes, libraries, CLIs)
• Solutions for running containers locally and remotely
• Helper tools and libraries to inspect, edit, or move images
• In-depth write-ups and video resources on the above.

And I'm hoping for your contribution!

Distroless Container Images

Over the past few months, I spent quite some time researching the topic of how to create decent container images. Distroless (1.0) is one of the ways. You can learn more about the limitations of the scratch base images and the pros and cons of the distroless (aka "scratch for everyone else") base images in the below Twitter thread or in a long-form write-up on my blog - What's Inside Of a Distroless Container Image: Taking a Deeper Look:

Ivan Velichko @iximiuz What Is a Distroless Container Image? 🧵 Go (programming language) is famous for its statically linked binaries. You can take a Go executable, drop it into a "FROM scratch" container, and call it a day. But there might be a problem (keep reading) 👇 pic.twitter.com/kskCI3rqCC September 5th 2022 242 Retweets 1,006 Likes

​

How To Extract Image Filesystem Without Running Any Containers

Collected a bunch of receipts in this blog article.

Containers Are Just Linux Processes

...is another popular misconception. Intrigued? Then read on!

Docker History And Evolution

For the digital archeologists (like myself), I called out on Twitter asking folks if the Docker's history is documented somewhere. The thread received lots of great replies (with some of them coming from people who are/were directly involved), shedding light on Docker's original architecture (monolith written in Go, essentially just a UX layer on top of LXC), its evolution (LXC's functionality got re-implemented in Go giving birth to, first, libcontainer, and then runc) and reshaping (containerd being factored out). Go check it out 😉

​

What I Was Reading

• ​LXC and LXD: a different container story - when a superior UX beats the superior tech. This article will make you feel that LXC and LXD are a viable alternative to Docker (in reality, Docker-like containers and the corresponding UX & ecosystem became the de facto standard). As an engineer, I'm on the LXC side, but as a pragmatic developer, I'm siding with Docker.
• ​Yet Another Brief History of container(d) - a related one. From the old-day VMs, through Linux namespaces & cgroups, to LXC, then Docker, libcontainer, runc, and containerd. Well worth a read if you feel like a history lesson.
• ​Extending Docker’s Integration with containerd - containerd was factored out back in 2015-2016, but the work continues even in 2022. The article is not about history, though. It's about the practical implications of this move. Docker starts using even more features of containerd. And this is good (at least) for two reasons: 1) more code reuse means better compatibility; 2) Docker's image dealing capabilities will start being enriched faster.
• ​Where are containerd’s graph drivers? - if you want to learn more about the above move - the story behind storage drivers.
• ​Vulnerability Management for Go - this is just great! A much-needed addition to any modern programming language.
• ​In Defense of Package Managers - (in the context of deplorable software security situation) Package Managers isn't the problem. The problem is how we write (with lots of bugs & vulnerabilities) and release (developers are quicker than distro maintainers) our software. LTS Linux distros can't keep up with the industry's pace. Is stream distros with frequent patches the future?
• ​What your scanner doesn't know **can** hurt you - continuing the above theme...
• ​HackTricks - a worthwhile resource on container security with some practical examples, including Docker containers breakout.
• ​Securing Containers with Seccomp: Part 1 & Part 2 - a highly-practical mini-series on how to use seccomp profiles to fine-tune your containers.
• ​Istio: Introducing Ambient Mesh - this does sound like a reasonable redesign to me! Migrating legacy workloads to Istio is not easy (I know it first-hand), but with the new basic L4 layer implemented on the ambient infra side, this initial step might become much easier. And since there is no need for sidecar injection anymore, the final steps will also likely be simplified. The single-tenant, scalable, and optional L7 proxy layer implemented as a bunch of Envoy Pods does sound like a good idea too. If, from the app's code standpoint, mesh remains fully transparent, these changes are a very strong reason to start liking Istio again!
  • 🎬 Istio Ambient Mesh Launch Demo - nice demo (only 10m), but you can skip the hands-on part and instead look at three diagrams that Christian Posta shows there. They might shed more light than the above Istio.io blog post.
  • ​[Kubernetes Podcast] Ambient Mesh, with Justin Pettit and Ethan Jackson - a lot of technical details, historical reasons for the redesign, insights on how meshes are actually used (mostly for mTLS), and how this new design was actually done by [virtual] networking people (vs. the sidecar-based design that was influenced by a background in application development).
• ​Why fix Kubernetes and Systemd? - Kubernetes vs systemd. Wait, what? But on second thought, it makes total sense. Kubelet's and systemd's responsibilities do intersect substantially. Kris Nóva continues the "war on sidecars" started by service meshes some time ago.

And that's all the links for today!

​

Stay Tuned

Well, I thought the previous one was big... I should probably start doing this newsletter twice a month, what do you think? Hit reply and let me know :)

​

Cheers

Ivan Velichko

​