270
Reddit DevOps. #devops Thanks @reddit2telegram and @r_channels
Mentor help for DevOps
Hello,
I currently have 4 years of experience ( bits and pieces in everything ) most in DevOps. Some in vulnerability fixes etc..
Started out fresh out of college. Got into a decent team old tech but good team. Kept working for a while and felt I could do it in IT. On my request I went to a different team which was DevOps work(still work for old sometimes).
Started out fine , I learnt many things I started delivering good works in short spans. Enjoyed the time. I felt intimidated many times because everyone in the new team were architects and leads with minimum 20 years experience. But pushed through, architects were great they were encouraging.
Things happened (project closed , architects left) so and soo and I was moved to a different section of the same project. I was now the CI-CD architect for this new initiative. Designed and implemented the CI-CD pipeline, the IaC infra etc.
But I don't know anything about the product per say or its business logic or anything to do any testing or anything of that sort. I get pulled into many of the architect meetings etc but I don't understand most of it anyways. Even when I see my ci-cd codes and iac code I feel like it's not upto the mark and is missing many basic items such as indentation or proper code structure. As I am always around big shots with 20+ year experience and I am the only person in the team to be with such less experience. I feel intimidated and feel like I don't know anything. Is this normal???
I have now been to a point where I am relaying more heavily on AI to do my task and code and I can't remember myself writing code at all.
How can I cope with this and skill up so that I am up to mark and upskilled to the current market.
https://redd.it/1uij1ma
@r_devops
What's your favorite AI agent harness/framework, and why?
Curious what everyone here is using these days.
• What's your go-to harness/framework?
• Why that one over the others?
https://redd.it/1ui4uyi
@r_devops
I want to learn More about PRs and Unit testing
How does your team handle updating unit tests after large pull requests? What's the most frustrating part?
https://redd.it/1ui199x
@r_devops
How do you make sure everyone on the team uses the same AI agent setup?
Before anything else: this is not a product pitch.
I’m genuinely trying to learn how teams are approaching this today, whether through OSS projects, internal tooling, shared repos, review processes, or just good old conventions.
How are you managing and aligning agent skills, rules, instructions, prompts, and MCP servers across your team?
Now, I want to tap into the community brain for a second.
As AI coding agents become part of the daily developer workflow, teams are starting to accumulate more than just code.
They now have:
Skills
Rules
Instructions
Prompts
MCP servers
Hooks
Agent-specific configurations
And at some point, this becomes a real coordination problem.
Now, instructions?
How do you keep skills and rules updated across projects?
How do you avoid every developer creating their own slightly different agent setup?
How do you manage trust when someone pulls an MCP server, prompt, or skill from somewhere online?
And maybe the biggest question:
Are you treating these agent primitives as something that should be versioned, reviewed, and governed or is it still mostly copy-paste and tribal knowledge?
Curious how teams are handling this today.
Are you managing it manually?
Using internal repos?
Documenting conventions?
Building tooling around it?
Or just letting each developer configure their agent however they want?
Would love to hear what’s working, what’s messy, and what you think this should look like as AI agents become more embedded in the SDLC.
https://redd.it/1uhva98
@r_devops
What should I do if higher ups think in silos?
I work in a European gov company for some years now and I really like it there. In my previous roles as DevOps engineer in different product teams I did my best to automate stuff within the boundaries of my team.
2-3 months ago I switched role to plattform engineer. During the interview for this new role HR asked me why I think they should choose me for the position. I answered that apart of being experienced with all the devops tooling I worked already in three product teams in this company and again and again did hacks for the same problems on teams scope for stuff that could and should have been solved on global scope and I know better than anyone the teams pain and I know how to solve it.
Well, I got the job and I really have dozens of ideas how to save lot of money, increase security while making product teams not have deal with security and lot more stuff but higher ups think in silos and “it’s their budget, not our budget” and “we will do exactly x. Not more and not less”
How would you deal with it?
https://redd.it/1uh78tp
@r_devops
devops browser game that uses AI to argue with you on your decisions unless you are confident
hi all
I built a browser game where you argue with AI on a given challenge/scenario and it rates your responses.
right now the scenarios are about devops/engineering, but I am planning to add interview kit, from 0 to hero, etc...
how it is different from just using chatgpt:
when you ask chatgpt for a scenario and then give your answer, it mostly agrees with you. it wants to be nice, so even if your answer is bad it says "good point" and you walk away thinking you did well. it also does not really know the correct answer, it just makes one up on the spot.
in my game every scenario already has a correct answer that i wrote before. the AI plays a strict senior engineer. it does not agree with you, it pushes back and tries to find the holes in your reasoning. at the end you get a score, and it shows what you got right, what you missed, and the real answer. so you can not win by just sounding confident.
why i think it is useful:
you find out if you are actually right, or if you only think you are right. you also practice defending your decision out loud, like in a real interview or a real incident at work. and the feedback is honest, not just "nice job".
how you learn from it:
you make a call, the AI argues back, and you see exactly where your thinking breaks. then it gives you the takeaway. so you learn from your own mistakes instead of only reading theory.
how it could teach from zero:
a beginner can start with the easy scenarios. when they answer wrong, the AI explains why and shows the right way step by step. so even if you know almost nothing, it can walk you through it like a patient teacher that keeps asking "why".
i am not sure if people would actually use this, so i wanted to ask:
would you try something like this? and for what topic (devops, coding, system design, interviews, something else)?
I am also considering using this as a main engine to challenge architecture decisions and solutions (basically you create scenario, give context and then have my AI argue until it makes sense)
thanks
https://redd.it/1uh4g28
@r_devops
DevOps: watching builds all day?
I'm not primarily a devops engineer, but whenever I do devops stuff, I realize I'm usually waiting for builds to complete and I can't easily switch to another task when things are building because something might actually happen that requires attention. How do full-time devops engineer handle this? I'm genuinely curious. I feel like most of the day is spend watching builds go through.
https://redd.it/1ugw6ud
@r_devops
fed 5 days of k8s logs into a 1m context model and it found the root cause of a cascading failure our team spent 2 days on
had an incident two weeks ago that took our team about 14 hours across 2 days to root cause. intermittent 502 spikes on our main api gateway, no obvious pattern, metrics all looked normal between spikes.
after we resolved it i decided to test something. exported the full incident window from our stack. 5 days of k8s pod logs from the affected namespace, prometheus metrics export covering cpu memory and network, the entire slack incident channel transcript, and jira comments from the postmortem. roughly 850k tokens total.
loaded all of it into minimax m3 and asked it to identify the root cause.
it found it in about 90 seconds. a cronjob running every 6 hours kicked off a heavy batch etl process. while running it consumed enough resources that the hpa started scaling up adjacent pods. when the cronjob finished the hpa scaled back down but the graceful shutdown period was set to 15 seconds while some long running requests needed 30 to 45 seconds to complete. those dropped requests queued up at the gateway and caused the 502 spikes.
the thing is this was exactly our conclusion. it took us 14 hours of bouncing between grafana dashboards, grepping through logs, and cross referencing slack threads to piece together. the cronjob to hpa to graceful shutdown chain was not obvious from any single data source.
i also tested a control question. asked about a container restart that happened on day 3 that was completely unrelated. it correctly identified that restart as a separate oom kill event with no connection to the 502 pattern.
not claiming this replaces actual debugging. the fact that i already knew the answer means this wasnt a blind test. but the speed of cross referencing 850k tokens of mixed signal data is something i havent seen before.
anyone using long context models for incident analysis or log forensics on their team
https://redd.it/1ugomad
@r_devops
DevOps or SAP Basis? Feeling stuck at a career crossroads
I've been working in IT infrastructure for several years, mainly on Linux systems, databases, production support, and enterprise applications. Recently, I've been thinking seriously about where I should specialize next, but I'm genuinely torn between two paths: DevOps and SAP Basis.
DevOps seems to have a huge ecosystem with skills like Docker, Kubernetes, CI/CD, cloud, automation, and SRE. It feels like a path with plenty of opportunities across industries and good long-term growth. On the other hand, SAP Basis seems to be a niche with fewer professionals, potentially less competition, and strong demand in large enterprises, especially if I eventually move into SAP HANA, cloud, or architecture.
My biggest priorities are:
Long-term career growth
Strong salary potential
Opportunities to work abroad
A career that's still relevant 10–15 years from now
I'm not looking for the "easier" option—I don't mind spending the next couple of years learning if it leads to a better career. What I'm struggling with is figuring out which path has the better return on that investment.
For those who've worked in either (or both), if you were starting from an infrastructure/Linux background today, which path would you choose and why? Are there any downsides or realities that people don't usually talk about?
I'd really appreciate hearing from people who have firsthand experience rather than just general opinions.
https://redd.it/1ug6v6a
@r_devops
Your AI agents are sending customer data to OpenAI on every run and your DLP doesn't see it
Security teams control what leaves the network. Firewalls, DLP, CASB. Then a developer builds a workflow that pulls CRM records and sends them to OpenAI in a prompt. Every execution. No log. Nothing in DLP because it's HTTPS to an approved endpoint.
Your DLP sees: traffic to api.openai.com ✓
What's actually in those requests: customer names, emails, whatever's in the record.
The system prompt instruction is not a control. It doesn't survive a real security review.
Is your security team even aware of the data flowing through automation workflows?
https://redd.it/1ug3ull
@r_devops
manual thing?
\- Do you trust cloud-native tools (Cost Anomaly Detection, Azure Advisor, GCP Recommender) for this?
\- Would you rather have this fail a pipeline or just report?
Tool is open source: https://github.com/cleancloud-io/cleancloud
https://redd.it/1ug0irq
@r_devops
Transitioning from .NET Dev to DevOps/Infra/ETL at a massive corp - Is it worth losing the "Dev" for overseas goals?
Hi everyone,
I’m currently at a career junction and would love to get some perspective from this community, especially from those who have transitioned between Dev and Ops or have successfully moved abroad.
My Background & Goal:
Current Role: .NET Backend Engineer at a mid-sized retail company
Credentials: Recently passed the aws SAA exam.
Ultimate Goal: I want to relocate and find a software engineering or DevOps role overseas in the near future (targeting countries like the UK, Germany, Netherlands, Australia, or Canada etc).
The Situation:
I’m feeling highly anxious about staying as a .NET developer due to how fast AI is proceeding, and I feel like pure application coding might have limited long-term defense as a career moat.
I currently have two paths forward:
1. Path A (Internal Move): Stay at my current company and ask my manager for opportunities to work on our existing AWS cloud infrastructure.
2. Path B (The Job Offer): Accept an offer from a financial company . The role is focus on DevOps, Infrastructure, and ETL.
My worries :
If I take Path B (the enterprise DevOps offer), I will likely have to stop touching .NET backend development entirely. I am a bit sad to leave development behind, but Path B offers CI/CD, data pipelines (ETL), and infrastructure management.
Given that my ultimate goal is to move overseas (UK/EU/Aus/Can etc)
Is full-time enterprise DevOps/Infra experience more marketable and "defensible" globally right now compared to mid-level .NET development?
Will losing the "Dev" coding aspect hurt my chances of moving abroad? Which opps provide better chances nowadays for foreign workers ?
For those in the UK/EU/Aus markets, how is the demand for DevOps engineers with a strong developer background vs. pure backend devs?
Would love to hear your honest thoughts. Thanks in prior!
https://redd.it/1ufxoa4
@r_devops
DevOps culture stuff
I know that DevOps has become a role now and I'm cool with that. There are a typical set of tasks we do that employers need done, so why not?
But what has become of the culture part of DevOps? Shift left. Fail fast. Break down silos. Etc. Have we achieved all those things and so we don't need to talk about them anymore? When people ask "How do I learn DevOps" do we just assume they'll pick up on the culture stuff on the job? Has the culture stuff moved to other tech management roles? Do those things matter anymore?
https://redd.it/1ufm9kn
@r_devops
What are DevOps interviews like?
I’ve been working full time for a year, but during that year I’ve been “motivated” to use Claude code to do basic code and while I understand the code, I forgot how to write code and never was a fan of memorizing leetcode to land a position.
2 days ago I got a call about an interview for a DevOps position and while all my friends who have had interviews never had an actual coding question given, but rather all scenarios and system design, I read online that a lot of interviews still put you on the spot and either ask coding questions or a practical question to do some networking or Linux configuration and while I know how to do all that, I usually research when I forget a command especially ones I don’t use a lot, and I’m not sure they’ll allow me Google during the interview.
so I wanted to know how the average interview goes and what should I study and focus on?
https://redd.it/1ufh5aj
@r_devops
Who's responsible for Fastlane, DevOps or the mobile devs
I've played a bit with Fastlane solo, but I'm wondering how it normally plays out at larger companies. Do the mobile devs handle the Fastlane scripts, or does it become a DevOps responsibility?
Got to love writing Ruby just for releases...
https://redd.it/1ufb53e
@r_devops
practical knowledge resources and roadmaps for linux
what roadmaps and useful material do you suggest for taking my linux knowledge to the next level if im not focusing on certs and just wanna improve my usable linux knowledge in dev/network field. i already work with linux and have somewhat beginner knowledge but just wanted to improve it in a funcinal/practical/applied way
https://redd.it/1ui5vty
@r_devops
Decent approach for Ray Cluster IaC for non-homogeneous GPU & resources
edit: no idea why being removed. Just asking since it's heavy Python if anyone has approaches for IaC of Ray.
https://redd.it/1uhzft1
@r_devops
How to Generate RED Metrics from Traces Without Blowing Up Your Cardinality?
https://telflo.com/blog/span-metrics-without-the-cardinality-explosion
https://redd.it/1uha28j
@r_devops
the state of the job market in 2026...
https://redd.it/1uh5qa1
@r_devops
Meta How valuable are the skills required to build platforms like a white label/saas style EMR website platform (WordPress) to people hiring for various tech positions.
Is jack of all trades master of none worth anything?
Involves:
Windows servers and workstations split into separate networks
Cloud skill
WordPress font and backend
https://redd.it/1uh3cye
@r_devops
Github Action issue
I joined an organization, that uses GitHub Actions that are self-hosted in EKS. Whenever a job is pushed, it gets stuck on the runner until another job is pushed, which forces the first one to run. Where can i start looking to fix the runner issue?
It's an ARC in EKS.
https://redd.it/1ugth0q
@r_devops
The "Rogue AI Agent" runaway cost nightmare. Are you guys setting up guardrails yet?
We’ve started piloting automated AI coding agents in a few of our internal repos to handle routine refactoring and boilerplate generation. The velocity is cool, but from an infrastructure and cost perspective, it’s giving me absolute anxiety.
Last week, we had an agent get stuck in an unhandled edge-case loop during a refactor. It kept modifying code, triggering a local test execution, failing, re-generating the code based on the error log, and trying again.
Because it was running autonomously over the weekend, it burned through thousands of tokens and ran up a massive API bill before anyone noticed the alert on Monday morning. If it had been hooked up to spin up ephemeral cloud environments for integration testing, it would have been an absolute catastrophe.
It made me realize that while we have great guardrails for *human* mistakes (rate limits, budget alerts, branch protections), our infrastructure isn't really ready for the sheer speed and scale of an AI agent spinning out of control.
I’m trying to figure out how to box these things in before management greenlights them for the wider engineering org.
* **API Token Caps:** Are you enforcing hard daily token or spend limits at the API gateway layer per agent/session?
* **Loop Detection:** Has anyone built or used tools that detect recursive AI behavior (e.g., the same agent account making 20 commits or API calls within an hour on the same branch)?
* **FinOps Alerts:** Do your standard cloud cost anomaly alerts catch this fast enough, or are you building custom real-time webhooks for LLM spend?
Right now, it feels like we're handing keys to a Ferrari to a teenager who moves at Mach 5. How is everyone else tackling FinOps and guardrails specifically for automated coding agents?
https://redd.it/1ugfhnw
@r_devops
Using AWS ALB + Entra ID to add SSO to apps that don't support enterprise authentication
I've recently been building more internal tools, and one thing I've noticed is that many self-hosted applications (or AI-generated internal tools) either have very basic authentication or none at all.
Instead of implementing OIDC or SAML in every application, I tried moving authentication to the infrastructure layer using AWS Application Load Balancer's authentication feature.
In my example, I used:
\- AWS ALB
\- Microsoft Entra ID (OIDC)
\- Uptime Kuma
but the same approach works for almost any internal web application.
Benefits:
• No application changes
• Enterprise SSO
• MFA via Entra ID
• Conditional Access
• Immediate access revocation when accounts are disabled
If people are interested, I can also share the step-by-step configuration.
Has anyone else adopted this pattern? Or are you using something like Cloudflare Access, OAuth2 Proxy, or another reverse proxy instead?
https://redd.it/1ug3rvo
@r_devops
Starting new chapter as DevOps manager
Hear me out. After 20+ years of working as senior individual contributor and technical lead, I am moving into DevOps management. I am joining new organisation, so I am at a disadvantage of not knowing absolutely anyone. It’s in banking. Team of \~10. I am both most senior DevOps manager and engineer, so I hold authority in both, at least as far as Platform Engineering goes.
What would your advice be in how to handle 1st day, 1st week, 1st month?
https://redd.it/1ug2kmd
@r_devops
Open sourced a CLI that catches idle AWS resources in CI/CD
Disclosure: I'm the author of an open-source tool in this space, mentioned below.
We've been running a read-only CLI scan as a GitHub Actions step that flags idle cloud resources before they pile up on the bill. 48 rules across AWS, Azure, and GCP \- here's the stuff that surprised us most:
AWS:
\- NAT Gateways — $32/mo to exist even with zero traffic
\- Idle RDS instances — 0 connections for weeks, full instance pricing
\- CloudWatch Log Groups — default retention is forever. Found teams storing terabytes they'll never read
\- Old EBS snapshots — accumulate silently for years
\- Idle SageMaker endpoints — paying for inference capacity nobody's calling
Azure:
\- Stopped-but-not-deallocated VMs — portal says "stopped", Azure says "pay me"
\- Load Balancers and App Gateways with no backends — Standard LBs bill regardless
\- Empty App Service Plans — paid tier, zero apps deployed
\- Idle Azure OpenAI provisioned deployments — PTUs allocated, zero requests
GCP:
\- Unattached Persistent Disks — left behind after VM deletions
\- Idle Cloud SQL — zero connections for weeks, still billing full price
\- Idle external IPs — $3.65/mo each, they accumulate fast
\- Vertex AI notebooks — left running after experiments, billing GPU 24/7
The pattern: OIDC federation (no stored secrets), read-only permissions, scan runs in CI, posts findings as a PR comment or fails the pipeline.
What I'm curious about:
\- Is anyone else running cost/waste checks as a CI step, or is it more of a periodic
Jr DevOps and AI
Hey guys, lately I've been feeling pretty down because I can't really do anything without AI, can't do a simple ticket.
Did my Bachelor's with AI, didn't even need to really think that hard for any project, did my first year of my Master's with AI
It's my first year working as a DevOps/System Engineer also my first year of working xD
I'm not against the use of AI, but I don't like this feeling of not knowing anything, or having a issue that I previously had and don't even remember how AI solved it.
Some of you may say that I can just work without it or Google it, but at the rate things want to be shipped because of AI I have no time to sit down a learn or spend time hitting my head against the wall or spending 30 minutes searching online.
I know you won't have a magical answer for my AI imposter syndrome but something that can help already goes a long way.
I hope you understand that I never did anything without AI nor on my college nor on work so I'm having a really hard time breaking out
Thanks
https://redd.it/1ufo3rb
@r_devops
Upcoming DevOps System Design Round – What Should I Expect?
Hey folks,
I have an upcoming DevOps Engineer interview that includes a system design round. Has anyone here gone through a similar interview recently?
I'd love to understand the kinds of questions, scenarios, or problem statements that are typically discussed during these rounds. Any tips, preparation strategies, or resources that helped you would be greatly appreciated.
To provide some context, based on the job description, the role is heavily focused on Kubernetes, CI/CD, and platform engineering.
Thanks in advance for sharing your experiences and advice!
https://redd.it/1ufhxln
@r_devops
Update on Project Yellow Olive: I added Kubernetes Deployment challenges to my Pokemon Yellow inspired TUI game
https://redd.it/1ufh0uz
@r_devops
DataDog alert(monitors) grouping
Hello!
I've moved to company that is using DataDog for storing logs, monitoring etc. Its not really that used in my team, so i tasked myself with some edits and showing possibilities.
I'm coming from company where i have used Grafana for monitoring and alerting, so i'm used to the system that grafana has for alerting - mainly for grouping etc.
Here, we have private location for Monitors, that is in our network and so can access internal resources. But, as it happens, local server might not be that reliable and last night had some outage. That triggered tens of monitors that are directly connected to synthetic http tests (so cant be configured manually, only by the original synthetic test), that were flapping on and off because of http timeouts. That made about 300 notifications in email in 3 hours.
Even that my team says this is really unique situation that didnt happen for at least 2 years, i would like to work with this problem and find solution that would solve this trouble, if it should come in the future. So, the first thing that came to my mind is grouping like in grafana, where if multiple alerts in one group trigger and alerts, only one notification will be sent, with summary of alerts. But it seems to me that DataDog doesnt have solution for it - the only closest thing is Composite Monitor, but that allows only 10 monitors to be in it. Tags and groups only work in single monitor, which isnt possible because of the synthetic tests. So is there any other possible solution? If anybody knows, i appreciate any help!
https://redd.it/1ufaina
@r_devops