Ivan Velichko

Ivan on Containers, Kubernetes, and Backend Development

Published almost 2 years ago • 4 min read

Hi, it's Ivan from

First off, I'm sorry I missed the last month's send-out, but for a few weeks, I simply couldn't pull myself together because of the situation in Ukraine and in... Russia. I have strong connections with both countries, so this is quite personal for me.

Can't say March's been a productive month for me either - the first few weeks my brain was rather half-dead, and there was absolutely zero motivation to continue working on the blog or pet projects with all the grim news coming from every channel. At some point, I was close to losing my faith in humanity completely - we were supposed to start fixing the climate together, but instead, we got into a new nuclear crisis...

But then I figured that being desperate is a road to nowhere. Regardless of the chances to succeed, we should strive to make this world a better place. Otherwise, we're already doomed. And my current way of contributing to this common goal is through my blog. Of course, my posts can't help stop wars, but they can help someone prepare for a job interview, or become a better specialist, or maybe find a passion for technology. Therefore, keeping my blog up and running has all the chances to improve someone else's life, not just mine.

So, before this newsletter turns into a personal diary, I'm stopping ranting and starting sharing tech stuff again.

What I Was Working On

After a few weeks of the break, I returned to the Working with the Kubernetes API series. In March, I published one new write-up on how Custom Resources, Custom Controllers, and Admission Webhooks, when combined, is just a fancy way to write new Kubernetes APIs. For some reason, it took me a while to realize it, even though it sounds obvious in hindsight. The kubebuilder project even states it explicitly in the project's description: "Kubebuilder - SDK for building Kubernetes APIs using CRDs." As always, it's been hidden in plain sight.

Research for the above post produced plenty of materials that can be used for a more practical spin-off article. The plan now is to prepare a tutorial on how to develop a new Kubernetes API. First, I'll manually follow all the necessary steps (registering a custom resource, configuring RBAC, writing a custom controller, setting up admission webhooks, etc.), and then I'll show how using the kubebuilder and controller-runtime projects can drastically reduce the amount of boilerplate code and speed up the initial phase of development.

Another project that got some love in March was my client-go-examples repo - a few examples were added/updated, including a mini-program showing how to use the work queue - a basis behind almost every Kubernetes controller.

Lastly, one piece from February that I haven't had a chance to share here yet - a blog post with a bizarre title: The Influence of Plumbing on Programming - a story about T-shaped I/O pipelines in Go - io.TeeReader and io.MultiWriter. Pretty handy stuff to write concise and efficient I/O-intensive code.

SPONSORED but 100% relevant. Check out this blog post by Teleport: Authentication (AuthN) and Authorization (AuthZ) Tutorial for Kubernetes - an overview of the building blocks to secure the Kubernetes API access.

What I Was Reading

A lot of Kubernetes articles this month.

  • Kubernetes Controllers at Scale: Clients, Caches, Conflicts, Patches Explained - a large collection of tips, tricks, pitfalls, and best practices for writing Kubernetes controllers. Not a good first blog post on the topic, but an invaluable piece of information for more experienced Kubernetes developers looking for ways to improve their code.
  • The Missing Kubernetes Type System by Daniel Mangum - “Kubernetes API as a [distributed] type system” and “who is supposed to write Controllers.” (spoiler: subject matter experts, and not specially trained controller developers). More philosophical than practical write-up, but definitely worth reading since this is an opinion from a person deeply involved in the development of Crossplane - a universal controlplane atop Kubernetes.
  • Amazing series Life of a Packet in Kubernetes (Part 1, Part 2, Part 3) - in particular, I like the chosen explanation order - it starts from lower-level container networking, then goes to a network fabric as a means of cross-node communication, and finally touches upon higher-level networking concepts like service discovery and network policies.
  • Mermaid - "Javascript based diagramming and charting tool that renders Markdown-inspired text definitions to create and modify diagrams dynamically." Seems like a nice library, and I'm already looking for a way to use it on my blog to make the content more dynamic (since it's generated diagrams, I could use some scripting to animate the end result). By the way, Kubernetes docs use it as well!
  • Sandboxing and Workload Isolation - a thorough overview of workload isolation means starting from chroot and ending with AWS Firecracker by folks.

Tech News I've Come Across

Until next time

Well, this is it for today. It's been a rough month, and it's not over yet, but I truly hope things will start getting better soon. Unless we're all nuked, of course. But until then, make code, not war!

Stay strong,

Ivan Velichko

Ivan Velichko

Software Engineer at day. Tech Storyteller at night. Helping people master Containers.

Read more from Ivan Velichko

Hello there! 👋 Debugging containerized applications is... challenging. Debugging apps that use slim variants of container images is double challenging. And debugging slim containers in hardened production environments is often close to impossible. Before jumping to the DevOps problems that I prepared for you this week, let's review a few tricks that can be used to troubleshoot containers. If the container has a shell inside, running commands in it with docker exec (or kubectl exec) is...

10 days ago • 1 min read

Hey hey! Are you ready for your next DevOps challenge? Last week, we all witnessed yet another terrifying cyber-security event, and this time, it was a direct hit - researchers from Snyk discovered a way to break out of containers! 🤯 The vulnerability was found in the fundamental component of the containerization ecosystem - the most popular implementation of the (low-level) OCI container runtime - runc. Notice how, on the diagram above, most high-level container runtimes actually rely on the...

29 days ago • 1 min read

Hello friends! Ivan's here - with my traditional monthly roundup of all things Linux, Containers, Kubernetes, and Server-Side craft 🧙 What I was working on After my announcement of the iximiuz Labs GA earlier this month, the platform's usage has more than doubled, so I had to focus on the system's stability and UX. As a result, I increased observability and test coverage, added one more bare-metal server, streamlined a bunch of use cases, and fixed a few bugs. The most notable user-facing...

about 1 month ago • 3 min read
Share this post