86
Reddit DevOps. #devops Thanks @reddit2telegram and @r_channels
Learning DevOps tools as Cloud Engineer. What are your recommendations?
I did a search on Reddit and thinking about the following options. Does anybody have recommendation for where to learn these DevOps techs?
1. IaC: Terraform or Ansible
2. CI/CD: Jenkins, or Github Actions
3. Containerization: Docker, and Kubernetes
https://redd.it/1foj83g
@r_devops
some questions on hashicorp nomad
hello new to nomad and have some questions.
assume everything on AWS.
1. is the multi region federation able to do [automatic\] disaster recovery if a region fails?
2. how are you doing ingress for workloads running in nomad for say webapps? just using ALB target group that points to nomad client agents? anything else?
3. how are you doing persistent volumes for nomad workloads?
4. CICD / as-code: is waypoint the best way? anything else?
thank you!
https://redd.it/1fog92q
@r_devops
Switching domains amd getting into DevOps
Hello All, I've been working in IT industry for 3 years now in Robotic Process automation. I mostly work using low code no code tools. Ans occasionally on python.
I'm looking for a switch into Devops / Cloud.
I am going to start with Abhishek Veeramalas Course on youtube. Is it a good start? Also can anyone Mentor me through this journey of mine!
Can guarantee that I can give 2 - 3 hours everyday to learn it!
https://redd.it/1fo9j45
@r_devops
Junior cloud
Dear Redditors,
I would like to ask for your help on how I could start a career in the junior cloud field. I'm interested in IT, but I have a degree in a completely different field (biology), so I would need to start almost from scratch. How could I set up a roadmap, and what should I study for this? Thank you so much for your help!
https://redd.it/1fo88r0
@r_devops
How hard it going to be to enter Cloud Platform Engineer/Devops. Read below
So in total, I have 9 years of experience.
The first 4 years I worked in a call center.
Then I got a Service Desk role in an IT firm. After 4 years landed 2 promotions and got the SysAdmin role. The stuff I do as a sysadmin is very basic or at least I think it is.
So now, I am finally learning az104 and scripting. And I think I am doing at the learning part.
My question is with the huge not so relevant experience I carry, how hard is it going to be get in as an Azure/Cloud Admin to begin with?
I always see atleast 4 years of relevant experience being asked on job descriptions so that's the reason this question.
I see people who out of nowhere(As in, without any prior experience)getting Data Analytics and Data Science with huge packages. Is the same possible on this space?
https://redd.it/1fo6ci6
@r_devops
How do you detect upcoming break changes in cmdlet?
Hi
I have PowerShell running in Microsoft Hosted Agents and last week I noticed the following warning:
WARNING: Upcoming breaking changes in the cmdlet 'Get-AzAccessToken'
Luckily I saw the message and could take steps to address the upbreaking changes. Otherwise, the Microsoft Hosted Agents would eventually be upgraded to that version with the breaking change and my pipeline would have failed and would have needed to rush on a fix.
How do you manage this scenario?
I know, option 1 if Self-Hosted agent and manage the version. But assuming I want to keep Self-Hosted agents... any ideas?
Thank you
https://redd.it/1fnsh2l
@r_devops
How to improve performance while saving upto 40% on costs if using actions-runner-controller for Github actions on k8s
I posted a few days ago about how actions-runner-controller was an inefficient setup for self-hosting Github actions.
Since then, we ran a few experiments to get data (and code!). We see an ~41% reduction in cost and equal (or better) performance when using VMs instead of using actions-runner-controller (on aws).
Here are some details about the setup:
- Took an OSS repo (posthog in this case) for real world usage
- Auto generated commits over 2 hours
For arc:
- Set it up with karpenter (v1.0.2) for autoscaling, with a 5-min consolidation delay as we found that to be an optimal point given the duration of the jobs
- Used two modes: one node per job, and a variety of node sizes to let k8s pick
- Ran the k8s controllers etc on a dedicated node
- private networking with a NAT gw
- custom, small image on ECR in the same region
For VMs:
- Used WarpBuild to spin up the VMs.
- This can be done using alternate means such as the philips tf provider for gha as well.
## Results:
| Category | ARC (Varied Node Sizes) | WarpBuild | ARC (1 Job Per Node) |
| ------------------ | --------------------------- | ------------------ | ------------------------ |
| Total Jobs Ran | 960 | 960 | 960 |
| Node Type | m7a (varied vCPUs) | m7a.2xlarge | m7a.2xlarge |
| Max K8s Nodes | 8 | - | 27 |
| Storage | 300GiB per node | 150GiB per runner | 150GiB per node |
| IOPS | 5000 per node | 5000 per runner | 5000 per node |
| Throughput | 500Mbps per node | 500Mbps per runner | 500Mbps per node |
| Compute | $27.20 | $20.83 | $22.98 |
| EC2-Other | $18.45 | $0.27 | $19.39 |
| VPC | $0.23 | $0.29 | $0.23 |
| S3 | $0.001 | $0.01 | $0.001 |
| WarpBuild Costs | - | $3.80 | - |
| Total Cost | $45.88 | $25.20 | $42.60 |
## Job stats
| Test | ARC (Varied Node Sizes) | WarpBuild | ARC (1 Job Per Node) |
| ----------------------- | --------------------------- | -------------------- | ------------------------ |
| Code Quality Checks | ~9 minutes 30 seconds | ~7 minutes | ~7 minutes |
| Jest Test (FOSS) | ~2 minutes 10 seconds | ~1 minute 30 seconds | ~1 minute 30 seconds |
| Jest Test (EE) | ~1 minute 35 seconds | ~1 minute 25 seconds | ~1 minute 25 seconds |
The blog post contains the full details of the setup including code for all of these steps:
1. Setting up ARC with karpenter v1 on k8s 1.30 using terraform
1. Auto-commit scripts
https://www.warpbuild.com/blog/arc-warpbuild-comparison-case-study
Let me if you think more optimizations can be done to the setup.
https://redd.it/1fnqpn9
@r_devops
Need a genuine guidance here
A few days back, on Thursday night, I had the hiring manager round. It went all good I hope. Answered all their questions, discussed my work, all..... But I'm doubtful that my education background might ruin it all.
So, actually my background is not so technical. I studied statistics and data science in my uni but when it came to hiring, I got the job hoping to get into DS or AI/ML stuff. But instead they assigned me to DevOps. I moved forward with it because I read that DevOps is more culture than just a technical skill, DevOps is an integral part of Data Science and ML engineering and blah blah blah....
So, they asked me why I changed to DevOps. I gave them an honest answer that I didn't know about DevOps initially, and when I went through about DevOps, I took the role as a challenge and I stayed because I liked doing DevOps stuff.
The rest of the interview was all about my skills and all the things I worked on, like writing scripts, Jenkins, CI/CD pipeline, security, IAM automation,etc.
It's Monday today, I haven't heard from them yet.
Did I ruin it all? Did I make a mistake for not knowing DevOps? Or Did I make a mistake switching to DevOps?
PS: I have 2 years of experience as a DevOps Engineer and I maintained good work at my current company (~4+ avg rating)
https://redd.it/1fnhmpb
@r_devops
Good linter for Dockerfile
Hello everyone,
For a project I need to use a linter for Dockerfile and add a lot of rules that aren't covered by classic tools (cyber rules essentially).
Hadolint (https://github.com/hadolint/hadolint) was a pretty good candidate but it has some point that annoy me a lot :
- It's pretty hard to add new rules (Need to code and compile in Haskell, documentation isn't clear about it)
- The project have a lot of issues and pull request without response (Complicated to add new rules)
The strength of hadolint is the use of Spellcheck and AST that make possible a deep analysis.
Other projects like dockerfilelint or dockerfile_lint seems dead.
Do you have any recommandation about Dockerfile linter where I can add rules?
I have thought about making another Dockerfile linter, but the main goal isn't to add a concurrent to the list.
Thanks!
https://redd.it/1fngvl4
@r_devops
DevOps course for small companies and individuals
Hello everyone,
I've posted this here before, but I've updated the course a bit based on student feedback, and I've also redid the GitLab Runner section since v17+ has a new way of registering runners.
The course is aimed at small companies and individuals who want to self-host a variety of services on a single VPS.
As for prerequisites, you can't be a complete beginner in the world of computers. If you've never even heard of Docker, if you don't know at least something about DNS, or if you don't have any experience with Linux, this course is probably not for you. That being said, I do explain the basics too, but probably not in enough detail for a complete beginner.
Here's a 100% OFF coupon if you want to check it out:
https://www.udemy.com/course/real-world-devops-project-from-start-to-finish/?couponCode=FREEDEVOPS2312PRPDC
Be sure to BUY the course for $0, and not sign up for Udemy's subscription plan. The Subscription plan is selected by default, but you want the BUY checkbox. If you see a price other than $0, chances are that all coupons have been used already.
You can try manually entering the coupon code because Udemy sometimes messes with the link.
The accompanying files for the course are at https://github.com/predmijat/realworlddevopscourse
I encourage you to watch "free preview" videos to get the sense of what will be covered, but here's the gist:
The goal of the course is to create an easily deployable and reproducible server which will have "everything" a startup or a small company will need - VPN, mail, Git, CI/CD, messaging, hosting websites and services, sharing files, calendar, etc. It can also be useful to individuals who want to self-host all of those - I ditched Google 99.9% and other than that being a good feeling, I'm not worried that some AI bug will lock my account with no one to talk to about resolving the issue.
Considering that it covers a wide variety of topics, it doesn't go in depth in any of those. Think of it as going down a highway towards the end destination, but on the way there I show you all the junctions where I think it's useful to do more research on the subject.
We'll deploy services inside Docker and LXC (Linux Containers). Those will include a mail server (iRedMail), Zulip (Slack and Microsoft Teams alternative), GitLab (with GitLab Runner and CI/CD), Nextcloud (file sharing, calendar, contacts, etc.), checkmk (monitoring solution), Pi-hole (ad blocking on DNS level), Traefik with Docker and file providers (a single HTTP/S entry point with automatic routing and TLS certificates).
We'll set up WireGuard, a modern and fast VPN solution for secure access to VPS' internal network, and I'll also show you how to get a wildcard TLS certificate with certbot and DNS provider.
To wrap it all up, we'll write a simple Python application that will compare a list of the desired backups with the list of finished backups, and send a result to a Zulip stream. We'll write the application, do a 'git push' to GitLab which will trigger a CI/CD pipeline that will build a Docker image, push it to a private registry, and then, with the help of the GitLab runner, run it on the VPS and post a result to a Zulip stream with a webhook.
When done, you'll be equipped to add additional services suited for your needs.
If this doesn't appeal to you, please leave the coupon for the next guy :)
I've shared this course here before - there's no new material, but I've brought few things up to date, and there are some new explanations in the Q&A section. Also make sure to check the annoucements, there are some interesting stuff there.
I hope that you'll find it useful!
Happy learning,
Predrag
https://redd.it/1fnekmp
@r_devops
How do you guys manage images in private network with no internet access allowed
I want to use private K8s cluster running across multiple on-prem servers with CI/CD applied.
so I deployed container registry and make helm refer private registry. but I'm wondering how does devops manage the dependencies of all theses images that a helm chart depends?
there are plenty of images online the helm chart can pull from the chart definition without making any fuss.
But how do you guys automate image supply chain up to date with the latest tags available in private network??
https://redd.it/1fncjue
@r_devops
Modern way to transition from KS into ansible
I have read interesting things from 6+ years ago, but there isn't too much literature on how people manage to kickstart bare-metal using ks files into Ansible.
Seems that one of the best approach a few years ago was to install ansible and git via post scripts, add a ssh public key and then ansible-pull the first configuration for the initial setup.
Technically that would work, but I wonder how people approach this problem today (also where sensitive creds are stored).
Cloud-init looks a cleaner way to go from ks into ansible, but it's extra piece to maintain.
https://redd.it/1fn15w8
@r_devops
If I want to host my SSR site as "ethically" and optimally as possible without using "big tech" solutions, what are my options?
I've already swapped Github out for https://codeberg.org, but when it comes to actual hosting, I'm a bit more lost. Is it possible to get an optimally served SSR site with good worldwide CDN coverage and all that without using big tech? If possible I would like to use European solutions.
Thanks!
https://redd.it/1fmqz5r
@r_devops
Termius has horrible security practices or am I overreacting?
I tried Termius in the past and it felt really bad that you need to have account and all your private keys will be synced with 3rd party, so I avoided this terminal like plague. However I got recently Yubikeys and decided to give it another shot, since Termius supports hardware keys this felt like a solution for the trust issue. Well I was wrong.
1. You can't use Yubikey as 2FA on Termius account. This was first major WTF for me. They support Yubikey for other purposes, but not on their accounts.
2. Termius Windows app does not lock after you close it or reboot your device. It won't ask for password, PIN or Yubikey to open the app. I don't think I need to explain why this is important. Funniest thing is that lock function is available on Android.
3. I can generate keys with Yubikey, that's cool. Right after generating and saving the key I can see my pass-phase, private key and public key. I guess if I close the app and re-open all the fields will be hidden and require pass/Yubikey to access it, right? RIGHT? Nope! All the fields are available right after you open the app.
During the key generation I didn't even think that there will be an option to view pass-phase, that it will be entirely stripped off the client.
So now to the real question, am I stupid or this is actually bad?
https://redd.it/1fmo10o
@r_devops
Which tools do you guys use to calculate subnets and keep track of them?
Hey everyone! 👋
I recently wrote a blog about one of the most useful tools I use for calculating CIDR ranges and subnets, which is the DavidC CIDR Subnet Calculator. It’s been a lifesaver when managing IP allocations for cloud environments and Kubernetes networks.
I’m curious to know—what tools are you all using for calculating subnets and keeping track of your IP ranges? Would love to hear about any alternatives or tools that have helped you with your network management.
I've written a blog about it here: https://www.dailytask.co/task/subnet-cidr-range-calculation-1726986261
Looking forward to your recommendations!
https://redd.it/1fmn1n7
@r_devops
what are best way and resources to learn ansible?
I'm fresher
https://redd.it/1foi7xp
@r_devops
Generative IaC from app code
Hi all,
Regarding Terraform IaC, I came across this company StackGen (formerly appCD) that generates Terraform code from application code (Python and Java) for new applications.
Does anybody know any competitors or have any additional thoughts on this? Seems very useful
https://redd.it/1fodnue
@r_devops
How to Balance Security with Speed in DevOps Pipelines?
I’m facing a challenge in balancing security checks (like static code analysis and vulnerability scanning) with the speed of our CI/CD pipelines. How do you ensure your pipelines remain secure without compromising speed?
https://redd.it/1fo7vba
@r_devops
How do you approach self-service with Terraform in an IDP?
Hey there!
I’ve been building platforms for developers with my teams using Terraform for a while now.
So far, our approach to self-service for developers with Terraform has been more or less to propose pre-made modules that are compliant with the org policies and propose sound defaults or are an abstraction (e.g an « app » module made of well-configured RDS, bucket, Fargate, etc).
All those approaches however always require you to somehow go through a PR and apply it via CICD etc
We are seeing more and more Internal Developer Portals (e.g Backstage, Port, etc) appearing in the landscape where now developers can have those « Boostrap a stack » buttons. Somehow, I guess this can leverage Terraform use your abstraction.
But how does it work state-wise?
Where is the « actual code », ie, the given module instantiation being written?
Is there an existing open-source way to make Terraform usable via an API?
All in all my questions are summarizing around: how can Terraform be made compatible via non-code way of working when it is code-oriented by design?
Cheers!
https://redd.it/1fo6gex
@r_devops
Interested in transitioning to Devops in the future.
Would you be willing to tell me more about it and is it worth pursuing.
just interested in it since it seems more aligned with my interests.
A little about me:
I’m a programmer; I’ve been programming since I was 7, mostly as a game mod developer for a GTA San Andreas server.
I moved on to Roblox when I was 13, where I designed scripts that manipulated server-side processes and created anti-cheats for games in exchange for Robux (not real money). I later transitioned to external exploits, semi-skidded, to be honest. Currently, I work as a freelancer since I'm a full-time student in college.
My only recent project is an unfinished Baldur's Gate mod compiler, but I stopped because many already exist without much room for diversity. My other project is a game I’m working on via Godot, which I picked up recently.
I plan to make mobile apps with it once I understand it better.
What got me into DevOps is the fast-paced industry, though I might go into DevSecOps due to my enjoyment of reverse engineering.
born in West Africa, came to the US as a baby so I have to be on that grind.
my biggest flex-> I have 10+ years of C++ programming experience before 20
im also extremely introverted, but i lack social awkwardness.
https://redd.it/1fnx66k
@r_devops
Take home assignment feedback [hiring]
My company is creating a take-home assignment for candidates to hire a DevOps engineer. Is this assignment too hard or reasonable? A pre-built repo will be given.
**1. Cloud Infrastructure Setup** (using AWS, GCP, or any cloud provider)
* Set up a **Virtual Private Cloud (VPC)** in your cloud environment to isolate your node infrastructure. Configure a private subnet.
* Deploy a **Dockerized application** (can be a simple Node.js or Python app) to an **EC2 instance** in the private subnet.
2. Set up a basic **CI/CD pipeline** using **GitHub Actions, Jenkins**, or another CI/CD tool of your choice. The pipeline should automatically:
* Build the Docker image.
* Push the image to a container registry (e.g., Docker Hub, AWS ECR).
* Deploy the latest version of the application to the EC2 instance when changes are pushed to a GitHub repository.
3. **Monitoring and Logging**
* Implement basic monitoring for your EC2 instance and the web application using **Prometheus**/**Grafana**, **AWS CloudWatch**, or any other monitoring tool
* Set up an alert to notify you if CPU usage exceeds 30% or the web application is unreachable
https://redd.it/1fnq8jc
@r_devops
Nginx Ingress OAuth2 infinite loop
I have a K8s deployment with NGINX Ingresses for a UI app and OAuth2 Proxy, which are behind the same load balancer. I can't load the UI app with the load balancer URL because a /oauth2/auth 499 infinite loop error occurs at the ingress controller, i.e. the request never reaches OAuth2 Proxy. What could be the problem?
Here are the nginx-ingress annotations:
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escapedrequesturi"
https://redd.it/1fnq39m
@r_devops
How Do You Handle Rollbacks in CI/CD Pipelines?
In our CI/CD pipeline, we’ve faced a few deployment failures that led to production issues. What are some effective strategies for handling rollbacks during deployment, especially when working with databases?
https://redd.it/1fnh7qp
@r_devops
DevOps/SysAdmin part-time
Hey guys,
There are remote part-time jobs for junior DevOps/SysAdmin or other Linux and INFRA based roles in US or EMEA?
Where do you find them? On LinkedIn I couldn't find.
Thanks!
https://redd.it/1fnh4i5
@r_devops
I just created a weekly newsletter for fully-remote, global, tech jobs
As a developer who regularly searches for remote tech roles, I started curating my own list of global, fully remote jobs that matched what I was looking for, and I've now decided to share it by creating a free newsletter: [fullremote.tech](https://fullremote.tech/)
Here’s what to expect:
* Only **Global/Async** jobs
* **Tech**-related jobs only (dev, AI, design, cybersec, data).
* I personally handpick the jobs each week.
* It’s **free**
* **No spam**, and I won’t share your email with anyone.
I’d appreciate any feedback or suggestions!
https://redd.it/1fnecfw
@r_devops
Work laptop
My friend used to work for tech company 1.5 yrs back and they gave him a hp zbook firefly laptop protected by vanguard security. They didn't ask for the laptop back. Can he use this laptop for personal use ? Thank you im advance
https://redd.it/1fn9eu2
@r_devops
How do you fight adhd while working on a project?
Adhd: attention disorder. Simply put, jumping from one thing to another, without completing anything
Basically, i have over 25-30 repos in my account. All of them
Started, when i felt excited about it, left it after couple of days and never completed anything.
In one year of experience as an sde at a startup, i wore a
frontend hat(inintiating a new project, setting up the basic template required and heading the move to new ui),
backend hat(working on entire etl pipeline and optimising all the apis),
devops hat(setup entire deployment pipeline),
sys admin hat(helping sys admin in moving developers laptops from windows to linux)
My personal projects in resume also reflect the same, UNFINISHED frontend, backend, devops projects.
I dont have a strong forte.
Now even if i want to shift to another company, i dunno whether i should
Prepare for devops/backend/fullstack
I like both backend and devops
How do you tackle this?
https://redd.it/1fmqj9j
@r_devops
Need help to understand more about Social media app Services & Storage expanse.
Hi Guys,
Need help to understand more about Social media mobile application running expanse. (Storage, services)
We are building a social media mobile application and while working on valuation, I calculated the expenses of running the system. (I'm calculating this for India Region)
But I have some doubts regarding the dynamic storage, seems I’m not doing it right. Can you guys help me to find a possible number?
below I’m sharing the per year user & their average storage.
1. What do you think about Static storage? For now, I’m exploring AWS EC2.
2. What do you think about Dynamic storage? For now, I’m exploring AWS S3.
3. What’s the best way to calculate the pricing for Dynamic storage? Assume below is 5-year user & per-user storage data.
4. Let me know if i'm missing anything.
* 1st year, 10000 user & 1 GB per user.
* 2nd year, 30000 user & 1.5 GB per user.
* 3rd year, 100000 user & 1.8 GB per user.
* 4th year, 35000 user & 2 GB per user.
* 5th year, 1200000 user & 2.5 GB per user.
https://redd.it/1fmno3b
@r_devops
is it naive to think that AI in future will help software engineers do devops?
I have 2 offers 1 in devops ($<) and 1 in software engineering both entry level. I want to go deep into devops but other subreddit post someone mentioned that AI will make everything easy and one person would be doing Software engineering + devops + qa. I have no experience have a say in this but would like understand on a deeper level.
https://redd.it/1fmo5z2
@r_devops
How are you managing your custom on-pre deployments?
UPDATE : please read ON-PREM in the title.
I worked in an environment where we managed lots of on-prem deployments on windows and currently on linux. We initially automated them with Perl and then using Powershell when gained traction few years back. Now we have started shifting things to Linux containers which would be eventually go to AWS on containers in next couple of years.
We have huge PS module (about 170+ complex functions) written in powershell and it’s difficult to port them into bash because- 1. Due to enormous amount of effort and 2. It will be useless in next couple of years as we go on cloud. Currently I am using POSH-SSH module to execute Linux commands from windows and only writing bash scripts where an interaction with the Linux OS is needed.
But I want to make this super easy for everyone with an Ops-Platform. But I can’t really figure out the time, effort and energy it requires. I know Ansible would help but for many bespoke reasons we have avoided adding config management tools into our ops repository.
I am confused but i need some suggestions…
https://redd.it/1fmlqx0
@r_devops