CatOps

17 December 2025 09:30

GitHub Actions will charge $0.002 per minute for self-hosted runners starting from the 1st of March 2026.

Obviously, you would still pay whatever you pay for your self-hosted infrastructure itself.

GitHub Actions will remain free for public repositories. For now.

#cicd #gha #microsoft

CatOps

15 December 2025 20:59

​​For today’s Donations Monday, let’s help Serhii Sternenko with his initiatives:

- Rusoriz - a standing Monobank jar. The goal is to buy 300 FPV drones daily.
- Fundraiser for the interceptor drones

#donations #Monday

CatOps

11 December 2025 17:56

Here's an article on using DRY and KISS principles when working with Terraform. In my opinion, this is one of those articles that has a good idea behind it, but lacks a bit in delivery.

KISS vs DRY in Infrastructure as Code: Why Simple Often Beats Clever.

The main take-way is, as usual: use your own judgment when creating abstractions for your infra code. This also applies to all your code.

I do generally agree on the tooling part. This is what Adam Jacobs called "A 200% knowledge problem": when adding an abstraction (a wrapper), you need to understand not only your code and the underlaying technologies, but also each layer of your abstractions. Thus, do not add wrappers unless you have to.

However, this article also touches an important point: you may feel like it's time to introduce an abstraction, but in reality, it's not.

#terraform #iac

CatOps

06 December 2025 11:53

At least Cloudflare is fast in sharing their postmortems.

https://blog.cloudflare.com/5-december-2025-outage/

A curious thing is this:

>>>
Customers that have their web assets served by our older FL1 proxy AND had the Cloudflare Managed Ruleset deployed were impacted. All requests for websites in this state returned an HTTP 500 error, with the small exception of some test endpoints such as /cdn-cgi/trace.
<<<

IIRC, in the previous incident on Nov 18, only the customers on the newer proxy version were impacted. So, one could say that Cloudflare had a single time-distributed total outage.

Another important thing:

>>>
Before the end of next week we will publish a detailed breakdown of all the resiliency projects underway, including the ones listed above. While that work is underway, we are locking down all changes to our network in order to ensure we have better mitigation and rollback systems before we begin again.
<<<

Honestly, looking forward to seeing the write-up. I can only imagine how stressed their team is after taking down a big chunk of the Internet twice in less than 30 days.


#cloudflare #postmortem

CatOps

04 December 2025 14:09

​​Ok, the bot is online again!

Yesterday, I watched a video from KubeCon NA by Denys Vasyliev (in Ukrainian), and at some point they were discussing the dusk of open source, because the major players shifted their focus towards monetization and proprietary solutions.

And just today, I learned that Minio (S3-compatible storage) has been moved into the "maintenance" mode.

Here's a discussion on Reddit about the alternatives.

#open_source #minio

CatOps

01 December 2025 10:39

​​For today’s Donations Monday, let’s help the foundations “Тихо” to raise money for FPV and Vampire drones.

https://send.monobank.ua/jar/WaFbzLzNK

This fundraiser was shared by a close friend of mine, so I trust it.

#donations #Ukraine

CatOps

26 November 2025 12:34

https://faun.pub/migrating-from-nginx-ingress-controller-to-kubernetes-gateway-api-using-ingress2gateway-c7e8cef212fa

CatOps

24 November 2025 16:19

​​For today’s Donations Monday, I would like to remind you about the foundation that we’ve been partnering with for DevOps Days Ukraine for years now.

UA Responders. Their specialization is medical equipment and such.

#donations #Ukraine

CatOps

20 November 2025 12:09

It's been a while since we had simple how-to articles here. So, here you are:
alipandidan/jenkins-jvm-monitoring-with-remote-jmx-c9477024d23b">How to enable the JMX port on Jenkins.

It's short and actionable, and you would be surprised to learn how many people use Jenkins till these days.

#ci #java #debug

CatOps

18 November 2025 18:31

We don't know why Cloudflare is down - their status page is not so detailed as one of AWS.

However, you can still check out some books on Humble Bundle:

- Data engineering & data science by O'Reilly.
- Software architecture by Pearson.

#books #bundle

CatOps

14 November 2025 14:24

A new issue of the CatOps Digest is here:

https://newsletter.catops.dev/p/catops-digest-2025-11-14

#digest #newsletter

CatOps

12 November 2025 17:27

For people nostalgic for on-premise setups, Dropbox reviled their new generation hardware setup and the challenges they face storing exabytes of data.

#on_prem

CatOps

10 November 2025 13:15

​​For today's Donations Monday, I'd like to share with you a fundraiser for the Optic Dragons unit - a specialized FPV drone assault unit of the 92nd Separate Assault Brigade.

They're raising funds for optical fiber drones, spare parts for converting drones to fiber optics, and supporting combat vehicles of pilot crews. The unit has been redeployed to the Pokrovsk direction where the situation is intense and they need more drone reels for optical drones.

Direct donation link:
https://send.monobank.ua/jar/7D7whfQHfF

Card number: 4441 1111 2291 2961

#donations #Ukraine

CatOps

07 November 2025 19:03

A book bundle that I wanted to post a couple of days ago, but forgot. So, here it is:

Linux for Professionals by Apress.

#books #linux

CatOps

03 November 2025 15:33

​​For today’s Donations Monday, I’d like to share with you a fundraiser for FPV drones from DeepState - a collective behind the close-to-real-time battlefield maps.

https://send.monobank.ua/jar/9AtiB8esqu

#donations #Ukraine

CatOps

16 December 2025 10:39

Cloudflare shares how they use Terraform in production.

Their setup is quite standard: Terraform, Atlantis, Conftest (OPA). One interesting thing is that they use their in-house tool called tfstate-butler to work around the lack of encryption of the Terraform states. Although, they do not disclose the details of this tool.

Another catchy quote:

>>>
...we do this at a global scale — where a single misconfiguration can propagate across our edge in seconds and lead to unintended consequences.

Yeah... We know, Cloudflare, we know...

#terraform #iac

CatOps

12 December 2025 18:29

A new issue of the CatOps digest is here!

https://newsletter.catops.dev/p/catops-digest-2025-12-12

#digest #newsletter

CatOps

10 December 2025 11:53

This isn't a technical article, but still an important one, I would say. This one is about the importance of making your work visible.

Shadow work in engineering teams.

For better or worse, in many companies, promotion cycle is the popularity contest, therefore you need to act accordingly.

This article is aimed at the managers, but you may find it useful as an individual contributor as well.

#culture

CatOps

05 December 2025 16:04

​​I don't know, when is the point, where we can all collectively agree that front-end frameworks have gone too far in their complexity.

Yet, here you are with the Cloudflare preliminary postmortem:

>>>
A change made to how Cloudflare's Web Application Firewall parses requests caused Cloudflare's network to be unavailable for several minutes this morning. This was not an attack; the change was deployed by our team to help mitigate the industry-wide vulnerability disclosed this week in React Server Components. We will share more information as we have it today.
<<<

https://www.cloudflarestatus.com/incidents/lfrm31y6sw9q

#cloudflare #postmortem

CatOps

03 December 2025 15:04

The bot I used for years to make posts into this channel has finally died. So, it seems like I won't be able to make neat buttons anymore :\

Yet, I have a couple of time-sensitive things for y'all:

- Cybersecurity books bundle by Packt
- Hacking book bundle by No Starch Press

Another time-sensitive topic: our friends at DOU are running their winter salary survey. More participants mean more accurate results, so jump in!

https://dou.ua/goto/rJks

#security #dou

CatOps

28 November 2025 11:28

A new CatOps Digest is here!

https://newsletter.catops.dev/p/catops-digest-2025-11-28

#digest #newsleter

CatOps

25 November 2025 17:33

Do you have the "What went well" section in your postmortems?

Here's an argument to have one with explanation of why this is important.

tl;dr: Because while each incident is different, there is a set of skills and behaviors that allow one to improvise under pressure to mitigate an incident. These skills and behaviors can be taught as well, and your "What went well" section is also for that.

#sre #incidents

CatOps

22 November 2025 13:11

Always Be Ready to Leave (Even If You Never Do) is not about keeping your CV up-to-date or socializing with recruiters, as it may seem from the title. It’s a short article on work habits that would keep you more efficient and, probably, happy at work; even if these habits would eventually make it easier for you to quit, if you choose to.

#culture

CatOps

19 November 2025 08:54

A postmortem from Cloudflare for yesterday’s outage is now available.

tl;dr:
>>>
The issue was not caused, directly or indirectly, by a cyber attack or malicious activity of any kind.Instead, it was triggered by a change to one of our database systems' permissions which caused the database to output multiple entries into a “feature file” used by our Bot Management system. That feature file, in turn, doubled in size. The larger-than-expected feature file was then propagated to all the machines that make up our network.
<<<

Another interesting thing:
>>>
Unrelated to this incident, we were and are currently migrating our customer traffic to a new version of our proxy service, internally known as FL2. Both versions were affected by the issue, although the impact observed was different.
Customers deployed on the new FL2 proxy engine, observed HTTP 5xx errors. Customers on our old proxy engine, known as FL, did not see errors, but bot scores were not generated correctly, resulting in all traffic receiving a bot score of zero. Customers that had rules deployed to block bots would have seen large numbers of false positives. Customers who were not using our bot score in their rules did not see any impact.
<<<

So, if you were not affected yesterday, you know why now.

#postmortem #cloudflare

CatOps

17 November 2025 16:01

​​For today's Donations Monday, I'd like to ask you to donate to the administrative needs of the "Come Back Alive" foundation.

It takes tremendous effort to run a foundation like this, and despite they can, they do not take money for the operational needs from regular donations. Thus, it's important to help them cover those needs as well!

https://savelife.in.ua/en/donate-en/#donate-fund-card-once

#donations #Ukraine

CatOps

13 November 2025 08:00

Press F to pay respects.

>>> Ingress NGINX Retirement: Kubernetes SIG Network and the Security Response Committee are announcing the upcoming retirement of Ingress NGINX. Best-effort maintenance will continue until March 2026. Afterward, there will be no further releases, no bugfixes, and no updates to resolve any security vulnerabilities that may be discovered. Existing deployments of Ingress NGINX will continue to function and installation artifacts will remain available.

Announcement page.

#kubernetes #nginx

CatOps

11 November 2025 09:36

An interesting lab for an overengineered solution from AWS for Kubernetes workloads right sizing.

Should you implement it this way? I don't know. But maybe, you want to play with GitOps, AWS Bedrock and all that stuff.

Also, it's funny how they say in the beginning that having VPA and Goldilocks inside a cluster is an overhead and additional management burden and then propose to create a cluster in GHA runtime and use generative AI to address that.

#aws #kubernetes

CatOps

09 November 2025 15:59

An article by Charity Majors on why thinking of Observability in pillars is limiting.

I recall a similar article from the past about how Facebook does their observability. It’s somewhere here on the channel.

The core idea is to treat all the signals as universal wide events that would allow one to preserve all the context and not hop between different tools.

#observability

CatOps

05 November 2025 08:30

More follow-ups for the AWS outage (Azure outage didn't generate that much press).

Lorin Hochstein analyzes the postmortem from the complexity point of view and comes to quite interesting conclusions that you can absolutely apply to your incidents and postmortems as well.

tl;dr is that incidents (especially bigger ones) are often unique. So, when reasoning about the preventive measures, you need not only to prevent similar incidents, but also get prepared to handle incidents in general, because the next incident may be not the same as the present one.

#reliability #sre #aws

CatOps

31 October 2025 17:45

A new issue of the CatOps Digest:

https://newsletter.catops.dev/p/catops-digest-2025-10-31

#digest #newsletter