5629
DevOps and other issues by Yurii Rochniak (@grem1in) - SRE @ Preply && Maksym Vlasov (@MaxymVlasov) - Engineer @ Star. Opinions on our own. We do not post ads including event announcements. Please, do not bother us with such requests!
An interesting point of view on reliability through the prism of everyday work and experience from other industries.
The normal work of creating reliability is an article by Lorin Hochstein, that asks: what instead of thinking of how an incident could have been prevented, we ask: what do we do daily to avoid having incidents constantly.
P.S. "Invert, always invert" - Carl Jacobi
#sre #reliability #culture
A (now) regular Thursday security advisory rubric.
"Fragnesia" is a newly discovered local privilege escalation kernel CVE from the same family of CopyFail and DirtyFrag.
It looks like the Dirty Frag mitigation (disabling the kernel modules esp4, esp6, and rxrpc) should help here as well.
#security
Enabling horizontal autoscaling with co-operative distributed rate limiting is an old article from Monzo that describes, how they built their internal distributed rate limiting solution.
The interesting part is the reasoning about whether a system works in an adversary environment (public facing) or not (internal system). The main question here: can you trust a client? The answer to this question influences the design a lot!
#system_design
A new issue of the CatOps Digest is here!
https://newsletter.catops.dev/p/catops-digest-2026-05-09
Happy Europe Day! 🇪🇺
#digest #newsletter
You may have heard already that Mitchell Hashimoto plans to move Ghostty away from GitHub.
It could be that you plan such a move yourself for whatever reason, but you're not sure yet. Here's a guide on how to push changes to GitHub and Codeberg simultaneously, so you could still keep the door open.
Codeberg is a non-profit European Git hosting. Although, this guide should work for any provider as long as you can use SSH keys for auth.
#programming #github
Figma has replaced PGBouncer with their own implementation called PGKeeper written in Go as a connection pooler for Postgres.
I really enjoyed this article, because they go into the implementation depths and describe why certain decisions were taken. Unfortunately, it doesn't always happen in such articles. Also, this is a nice reminder that software engineering is not only about writing CRUDs.
Unfortunately, they do not plan to open source it for now, also because it's too tightly coupled with libraries and approaches Figma uses internally. To be honest, it makes sense for in-house software to aim to one's specific needs rather than being generic enough to be open sourced.
#databases #postgres
An interesting application for eBPF: patching your Kubernetes nodes against CopyFail (see the previous post) with a DaemonSet. It is supposed to work even if the algif_aead module is built into the kernel.
https://github.com/iwanhae/copyfail-ebpf-k8s
#security
If you're hosting GitHub Enterprise Server, you need to update to address a recently discovered CVE.
What's interesting about this CVE is that it is a legit CVE that was discovered with AI. As WIZ researchers put it in the related article
Notably, this is one of the first critical vulnerabilities discovered in closed-source binaries using AI, highlighting a shift in how these flaws are identified.
For today’s Donations Monday, I would like to remind you about a smaller fundraiser that I posted several weeks ago. It’s moving, but rather slow. So, let’s boost it!
A fundraiser for radio-electronic equipment for the 25th Brigade.
Monobank jar:
https://send.monobank.ua/jar/5cXWfFMLHR
The fundraiser is 60% complete.
#donations #Ukraine
The Laws Of Architectural Work is a short article with two important insights about architectural decisions:
- They always come with trade-offs.
- Context matters.
This reminded me of a university professor from back in a day. He used to say: “There is no good solution, there is only an optimal solution for our case”. This phrase pretty much summarizes the whole premise of this article.
In any case, I think it’s an important reminder, taking into account that we can “outsource” more and more coding work, so what we left with is basically architectural work, being it software, infrastructure, networks, or something else.
P.S. This article was written in 2020, so take it into account, when you encounter words “recent” there. I’m digging through my archive of saved articles.
#architecture
Do you trust your colleagues?
An article Stop Using Pull Requests from the same author as the previous article in the channel, argues that they may be not ideal.
The core argument is that pull requests were originally created for low trust open source environment, in which contributors may have never seen each other, and often do not know each other at all. Development teams in the corporate world operate on another set of assumptions.
It's interesting that this article also builds up on the ideas of Thierry de Pauw. IIRC, I already posted his talk "Non blocking Pull Requests" on the channel, but in any case, I can do it again.
The main premise of the article is that you need to adopt T*D practices: test-driven development, trunk-based development, and another made-up T*D practice that basically means pair-programming.
From my experience I can say, that eliminating pull requests is probably not something you can do in a short run, but measuring the waiting time before PRs are merged is a good practice. Another good practice is to team-up on tasks or projects. So, basically pair-programming, but several people can still work on different tasks within a project, share context on this project, and thus be able to review each other's work almost immediately without much context switching.
T*D practices are also nice. Honestly, I have an impression that the majority of people are using the trunk-based merge model and continuous deployment these days. Also, it's interesting how AI can facilitate test-driven development: spec (by human) => test (by a machine) => tests review (by humans) => coding (by a machine).
#culture #programming
Yet another article on the topic of technical debt.
It uses Martin Fowler's "Technical Debt Quadrant" to reason about the technical debt and provides some advices on how to address it.
P.S. There is some self-promotion in the end, but it's subtle. So, I would say that this article is still a nice entry point into the topic.
#culture
As you may know, LocalStack deprecates their community version starting from the 23rd of March.
So, here are a few alternatives you may consider:
- MiniStack - a free alternative written in Python.
- Floci - another free alternative written in Java.
- Kumo - a lightweight AWS simulator written in Go.
I haven't tried any of them yet. I guess, I'll need to, since I'm using LocalStack to test my open sourced Terraform module.
#aws #localstack
Terragrunt has released version 1.0. According to them, this is not about a lot of brand-new features, but a commitment to backwards compatibility within the 1.x branch.
The press-release also has an overview of some features that Terragrunt has.
#terraform #terragrut #opentofu
A new issue of CatOps Digest is here!
https://newsletter.catops.dev/p/catops-digest-2026-04-04
#digest #newsletter
Continuing with security advisory.
NGINX ngx_http_rewrite_module vulnerability CVE-2026-42945.
~
NGINX Plus and NGINX Open Source have a vulnerability in the *ngx_http_rewrite_module* module. This vulnerability exists when the *rewrite* directive is followed by a *rewrite*, *if*, or *set* directive and an unnamed Perl-Compatible Regular Expression (PCRE) capture (for example, $1, $2) with a replacement string that includes a question mark (?). An unauthenticated attacker along with conditions beyond its control can exploit this vulnerability by sending crafted HTTP requests. This may cause a heap buffer overflow in the NGINX worker process leading to a restart. Additionally, for systems with Address Space Layout Randomization (ASLR ) disabled, code execution is possible. (CVE-2026-42945)
ngx_http_rewrite_module (and it's widely used!), you are likely vulnerable.
Consistent Hashing in 1 diagram and 198 words is a nice primer on the consistent hashing technique. Obviously, it doesn't go deep on the implementation or examples.
That Substack has some other nice primers as well. Some are good, others are not so much, but all of them could be a good start for a new topic.
#programming #primer
For today’s Donations Monday, I would like to remind you about one of the smaller fundraisers from the recent digest.
- Radio-electronic equipment for the 25th Brigade.
It’s more than 80% complete, and I’m sure that with your help, we can close it this week.
#donations #Ukraine
Remember copy.fail which we all checking a week ago?
Here is a continuation - another Linux 0-day to root.
https://github.com/V4bel/dirtyfrag
Btw, I can recommend to checkout /channel/setenforce_1 - channel fully dedicated to security, or better say - to vulns that will have real effect on you. No bullshit about "10 common vulns" which you can check on OWASP etc. Love it.
#security #linux
A book bundle on Linux, shells, and other OPS topics by O’Reilly on Humble Bundle.
Just beware that this is a reoccurring bundle. It was featured before, including on this channel. Double-check before you buy!
#books
My friends started a helper jar for two pickup trucks for AFU.
More info: https://www.instagram.com/p/DXpgaaWgH00
Monobank jar: https://send.monobank.ua/jar/3U1hBa5WPp
#donations #Ukraine
https://copy.fail/
Basically, you need to patch/recreate with new version everything that interact with not trusted part of internet.
Bug found was AI-assisted, btw.
#security
Apparently, the Dutch Central Bank is opting for the Lidl cloud instead of a US solution as their cloud provider.
Right now, digital sovereignty may sound like loud promises, but this is one of the main issues the European tech sector will have to solve in the nearest future.
P.S. It’s also a bit funny that a grocery store is completing with a book store in cloud computing.
#cloud #lidl
A new issue of the CatOps Digest is here!
https://newsletter.catops.dev/p/catops-digest-2026-04-24
#newsletter #digest
After painful and not particularly successful adoption path of IPv6, a draft proposal for IPv8 is here. However, it has some critical flaws not on the technical, but on the operational and policy level. In short, in its current form, it would make the Internet more prone to be controlled by a centralized entity.
Here’s the draft itself I haven't read it yet, but now I sure will.
#networking
This article may upset some people, but this is a very good retrospective on measuring developer productivity, and what the new AI era may mean for this.
The Developer Productivity Trap is a rather long article, but it totally worth the time! Especially, if you work on the “development experience” side of things or is responsible for engineering metrics. It’s especially valuable read, if you’re on the journey of implementing AI assistants in your company.
#devex #culture
For today’s Donations Monday, I’d like to ask you to help with another smaller scale fundraiser for radio-electronic equipment for the 25th Brigade.
Monobank jar:
https://send.monobank.ua/jar/5cXWfFMLHR
The fundraiser is ~41% complete for now.
#donations #Ukraine
The Comforting Lie Of SHA Pinning is an article inspired by those supply chain attacks that happened lately.
It shows some quirks of how GitHub works with SHAs, which are quite unexpected. The gist and the main excerpt:
From the platform’s perspective, a fork is a separate repository with a shared object graph/history. When the runner resolves the reference, it ultimately looks up the commit in the Git object database; if that object exists and is reachable, it can be used regardless of which fork introduced it. A commit object is globally identifiable. If the SHA exists anywhere reachable, that is apparently sufficient.
From time to time, I share a standing jar for FPV drones for a guy from my wife’s hometown.
Today, I’d like to share a fundraiser for rehabilitation of his brother-in-arms, who lost his leg near Kostiantynivka. Now he needs to undergo a series of surgeries. Here’s a Monobank jar to help him financially:
https://send.monobank.ua/jar/5AmpbpVRxm
Card number:
4874 1000 2602 4938
#donations #Ukraine
Linux - The Good Stuff is a book bundle by No Starch Press that really has good stuff! Including the book I recommend to everyone starting with Linux - "How Linux Works" by Brian Ward and "The Linux Programming Interface" by Michael Kerrisk for those who want to know how Linux works, but on the API level.
There are some other interesting books as well. Yet, this bundle is not cheap: you have to pay at least €56 unlike the usual €20-25 to unlock it.
#books #linux