Hello friends! Ivan's here - with another well overdue roundup of all things Linux, Containers, Kubernetes, and Server-Side craft 🧙 This time, I have a great excuse - in April, we became parents of a lovely little boy. Expectedly, he immediately made us overly busy for a few weeks, but no complaints! I wouldn't trade this business for anything else in my life ❤️ Now, to more technical news. What I was working onMy main focus remains on iximiuz Labs 🚀 Content authoring redesignIn March & April, I had to put a lot of effort into separating the content from the code. Historically, I used the Nuxt Content plugin and kept my content markdown files and the actual platform's code in the same repository. This allowed me to cut quite a few corners and start shipping the content faster. In particular, I could rely on the dev version of the platform for content editing and benefit from Nuxt's default hot-reloading mechanism. Over time, though, the initial quick win turned into an annoying limitation. Spinning up the dev version of the platform (a non-trivial process, by the way) just to work on a piece of content, re-deploying the whole app just to fix a typo in a blog post - things like that started negatively impacting my motivation to work on content. As we all know, DevEx matters - the longer your inner dev loop, the more flaky your CI, the less your willingness to write good, well-tested code. And apparently, the same is true for the AuthorEx! The last straw was a few really bright individuals offering me a hand with content production. Obviously, I couldn't expect them to spin up their own dev versions of iximiuz Labs to work on content. So, refactoring it was. Long story short, I had to find a way to host the raw content materials on some remote storage and allow people to use the production version of labs.iximiuz.com to create and edit content, hot-reloading included. As a result:
Did it change the "Author Experience"? Hell, yes! The inner loop shortened drastically, and the requirement to run a dev version of the platform is gone. But an interesting by-product of the above rework is that every premium tier user of the platform can now create tutorials, challenges, and even courses! It's not a well-documented feature (yet), but technically, it's already available. Beware, though, that by default, this content is for personal use and won't be accessible to other platform users. I still have to figure out the copy-right concerns, content sharing, and review workflows, but I'm already happy to work with individuals if they see fit for this functionality in their work (trainers, course instructors, dev rel folks working on product demos?) Other iximiuz Labs improvements
Get more powerful playgrounds, unlimited egress traffic, and full content access with iximiuz Labs Premium.
Does your company have a learning and development budget? Then this expense most likely can be reimbursed.
What I was writing (and not only)Building a Tool to Debug Minimal Container Images is a recent KubeCon talk by Kyle Quest and Saiyam Pathak on how to build a The below Twitter thread is a good starting point:
If you check the above materials and decide to put the new knowledge to work, I've carefully crafted a few container debugging challenges - do check them out. Last but not least, I finally fell in love with Dagger! ❤️🔥 The tool has always been on my radar, but mainly because of its exceptional team and the noble end goal - to fix our CI, once and for all. Despite my interest, I couldn't really find a good way to use the solution itself. However, the recent release of Dagger Functions changed everything! Things have finally clicked for me. I spent way more time than I'm willing to admit digging into the internals, reading all available materials, and migrating some of my personal projects to Dagger, and while the beginner's perspective is fresh, I decided to use it to my advantage. What Is Dagger and Why You Might Want to Use It is the first lesson in the mini-course I'm working on (and two more lessons are already in the works). I also prepared:
Hope you'll enjoy it! What I was readingCatching up on my reading list, finally! 🎬 A 10-year Detour: The Future of Application Delivery in a Containerized World (a.k.a “Dude, Where’s My Platform?”) - someone finally said it! Well, not someone - Solomon Hykes himself. The idea that you can offload all infra concerns to a cloud provider or a fancy PaaS and focus only on the app business logic is a utopia. Much like with physical goods, where building a factory is often an integral part of the development and production, working on a software project should include building and owning the “factory” - i.e., the platform. Cloud providers and PaaS can help with it, but it’s you who owns the factory, so you better embrace it. This will impose constraints on the project’s design, but there is absolutely nothing wrong with it - it’s just the reality. Don’t get hit with the pendulum: DevOps shifted too far left - I absolutely love the title. The idea that developers should fully own Ops as part of their responsibilities (look, we built all these great tools to make Ops easier for you) always sounded unrealistic to me. The DevOps movement shouldn’t have been about the full shift to the left. As with any other domain, the balanced approach is vital, and the optimal state lays somewhere in the middle. BuildKit in depth: Docker’s build engine explained - a not too deep/not too shallow read on BuildKit. Since Docker 23.0 (~March 2023), BuildKit is the default image build engine, so it’s a good idea to understand a little how it builds images under the hood - it may help speeding the builds up. What is Buildkit? - another great post on BuildKit, this time by Adam Gordon Bell of Earthly. I’d recommend it as a complementary (and much more hands-on) material to the above blog post. How ima.ge.cx works - look ma’, a real-world example of a fully serverless architecture! But the actual reason why I’m sharing this post is the Onboarding roulette: deleting our employee accounts daily - “I’m a strong believer in automated tests,” but “…it can take 30 minutes to create [a feature], but might take hours or days to establish automated tests for it.” That’s why I keep saying that balance is important, especially for smaller companies. “Product creation requires the art of strategically taking on debt. The “loan” allows teams to validate quickly, and discovered value can later be spent paying down the tech debt (with interest). This is true for startups raising venture, and it’s true for teams building MVPs. Spend too much time building expensive tests up front, and you may run out of time to ship, learn, and pivot.” The alternative testing approaches are as valid as automated testing, can be cheaper to implement, and they can provide real verification of the correctness and not just assertions of some conditions in code. “Keep in mind that “no automated tests” doesn’t mean “no testing at all.” […] Production traffic and thorough alerting can serve as a weak replacement for automated testing. […] Better than production traffic is a subset of production traffic (canary rollouts). […] Better than canary rollouts is dogfooding.” Benchmark results of Kubernetes network plugins (CNI) over 40Gbit/s network 2024 - a solid piece on Kubernetes CNI performance. For a while, I’ve had a gut feeling that unless you’re in a highly specialized domain, the performance of the CNI shouldn’t really matter, and this article kinda confirms it. You should probably focus on the CNI’s UX and feature completeness while pondering the options. Kubernetes 1.30: Beta Support For Pods With User Namespaces - the state of User Namespaces support in Kubernetes. The TL;DR is they’re likely coming relatively soon, but containerd and runc, the two most widely used container runtimes, still don’t support them fully (CRI-O and crun already do). Bonus: the blog post also has a video nicely demonstrating the recent container breakout vulnerability, making it relatively easy to understand. Istio Ambient is not a “Node Proxy” - a nice illustrated summary of the (at the time) new Istio Ambient architecture. I particularly liked how the architectural differences are described from a historical standpoint, adding an evolutionary aspect to the picture. Spinning YARN - A New Linux Malware Campaign Targets Docker, Apache Hadoop, Redis and Confluence - yet another campaign, nothing really new, but the post itself is pretty good. It demonstrates a bunch of anti-forensics and general system-weakening techniques used by the involved malware after it made its way to the host. Clearly, checking your shell history file or the List Of Top 8 Service Catalog Tools - not every company is big enough I, Cyborg: Using Co-Intelligence by Ethan Mollick (again) - a bunch of practical advice on how AI can be currently used to produce good writing (no, it’s not about generating articles with AI) and, maybe even more important, facilitate reading. The latter is well-aligned with my vision for iximiuz Labs to be a tool for thought and not just a “blog with playgrounds.” I’m already thinking of the possible way to incorporate AI into the student experience. How LLMs Work, Explained Without Math - an engineer explaining LLMs to engineers. Enjoyed the larger part of the post (the ending gets a little bit too intense with too many fancy terms being thrown at the reader). What’s worth learning if we have AGI? - a Patreon post by Andy Matuschak [an edu researcher whose work motivated me to build iximiuz Labs], which I cannot recommend reading highly enough. It’s about picking your learning strategy in the world of powerful enough AI (I’m not certain about AGI - this one can put everything upside down, perhaps more like ChatGPT on steroids). The author argues that in many areas, including software development, to solve a problem means to improvise while composing solutions for subproblems and “reflecting in action”. While AI may excel in solving every individual subtask (e.g., generating a piece of code or refactoring a code base), the end solution is often not known at the beginning of the work and may not be possible without the original and idiosyncratic vision of the engineer. The AI helps realize this vision much faster, so the engineering job will include more and more “steering” of the AI [agents]. But how can you steer the AI if you haven’t gained your own technical expertise and formed your own repertoire of moves? Thus, AI or not, being fluent in your domain and seeing the conceptual patterns behind implementation details remain the vital qualities of a successful engineer. But to acquire them, you have to spend enough time building systems without over-relying on the “magic box” of AI. More hands-on, more focusing on fundamentals, more composing bigger blocks, and much less about memorizing particular commands, flags, and syntax details. Wrapping upWorking on iximiuz Labs is a fun challenge for me personally, but I also keep hearing how the platform helped someone master a new skill or learn a new technology ~daily. With 8,500 registered users and 40,000 virtual machines fired up, the usage of the platform keeps growing, and I'm happy to put more love into it. However, the sustainability of this effort remains my biggest concern. The free tier is truly generous, but I do encourage you to try the premium tier. It comes with a 7-day free trial, so you'll get the taste of the much faster and more durable VMs, unlimited egress, and other yummy premium features. And if you find it useful, keep the subscription going and help me make the platform even better 🙌 Cheers Ivan |
Building labs.iximiuz.com - a place to help you learn Containers and Kubernetes the fun way 🚀
Hello friends! Ivan's here with another monthly roundup of all things Linux, Containers, Kubernetes, and Server Side 🧙 The issue's main topic is iximiuz Labs' largest-ever upgrade: Fresher and more streamlined look of the frontend UI 💙 A new 5.10 Linux kernel built with nftables support (finally, we can try out kube-proxy's nftables mode). New default playground user - laborant (yep, rootless containers learning for). New playgrounds: Ubuntu 24.04, Debian Trixie, Fedora, and Incus (yay! more...
Hello friends! Ivan's here with a slightly delayed July roundup of all things Linux, Containers, Kubernetes, and Server Side 🧙 What I was working on This month, I got nerd-sniped by cgroups. It all started when I ran into a pretty significant difference in how Docker and Kubernetes handle the OOM events. When you limit the memory usage of a multi-process Docker container, the OOM killer often terminates only one of the processes if the container runs out of memory. If this process is not the...
Hey there 👋 I spent a few weeks deep diving into cgroup v2, and I'm happy to share my findings with you! Everyone knows that Docker and Kubernetes use cgroups to limit the resources of containers and Pods. But did you know that it's very easy to run an arbitrary Linux process in a cgroup using much more basic tools? The only kernel's interface for cgroups is the virtual filesystem called cgroupfs typically mounted at /sys/fs/cgroup. Creating folders there and writing to files in them is...