86
Reddit DevOps. #devops Thanks @reddit2telegram and @r_channels
How can AI integrate to DevOps solution?
AI is growing rapidly and I don’t wanna be exclude from it. As a DevOps engineer, is there any practical usage of AI in DevOps?
Can anyone share some examples that how your company integrates AI into pipelines, workflows, or anything?
https://redd.it/1bpg7mj
@r_devops
Is there anyone who needs a Coursera Plus?
I will be inviting you to use Plus for a year (worth $399) on your email (corporate invites) at $39 and obviously you won't be paying me without any proof that you require from and before you are satisfied. If anyone is needy and actually needs it, can dm me. I'll help them!
https://redd.it/1bpb36x
@r_devops
Can we ban the low-effort “how do I go into DevOps” posts?
I see at least a couple of them a day and the answer’s always the same. If you can’t be bothered to do a basic search of the sub then you’re probably not gonna make it in DevOps anyways; this field requires a self-starter that knows how to find information.
https://redd.it/1bp9424
@r_devops
Containerized POS - Lipstick on a pig?
To preface this post, I'm new to all things DevOps but I definitely see it as the future of all IT in some facets.
We use Oracle Simphony as a POS system in our restaurants. Everything runs on windows. Currently we have onsite hyperv servers that host a ton of windows 10 VMs with Simphony installed on them. Then people RDP into those VMs from tablets.
I want to find a better way. I was inspired by Chick-fil-A's restaurant compute system
https://medium.com/chick-fil-atech/enterprise-restaurant-compute-f5e2fd63d20f
And id love to build something much more scalable, resilient, and overall better. Simphony can be installed on Linux and would theoretically move to cloud virtual machines or even containers. My only fear with that is ensuring the restaurant can operate without Internet.
I found this repo for a kubernetes vdi
https://github.com/webmeshproj/webmesh-vdi
But it definitely don't want to be using that in a production env
Then that brings up the question of, is this even worth it, possible, or am I putting lipstick on a pig? We don't have an internal dev team (even though we should) so it just seems a bit overkill since we use an out of box solution.
Thoughts?
https://redd.it/1bp5gib
@r_devops
Terraform 1.7 Adds Config-Driven Remove and Test Mocking Ahead of OpenTofu
https://www.infoq.com/news/2024/03/terraform-test-mocking-remove/
Hashicorp announced the release of Terraform 1.7, a new version of the popular Infrastructure as Code (IaC) tool. Terraform now supports config-driven remove capability, a safer way to remove resources from the managed stack’s state data. The new version also comes with mock providers and overrides, as well as several other enhancements in the test framework.
https://redd.it/1boxtzt
@r_devops
Fullstack dev tasked to do some devops : looking for pointers to learn what i should know.
Hello r/devops and sorry if my title is too vague, i'll enter in the details.
So, I'm part of a small dev teams, 5 people, and until today we were self hosting all of our tools : Jira for out ticketing system, Jenkins for our Continuous Integration and unit testing, perforce -soon to be git- repo. We're tech-savvy and can rtfm so we set up the vms to host these services ourselves, but it's in no way our job or our domain.
2024 is the year of the migration to the cloud for us as asked by the management. We already migrated Jira and our repo should be done by the end of the year.
I've been asked to look for solutions to do the same with our CI. The complexity seems a level higher as CI is not "static" in the sense that it needs to pull files, to have sdk installed to compile them (we do .Net6), to run unit tests on the result and copy it to a place where we can access it, all while keeping the interactions between Jira and our CI for version release for example.
And of top of that, there's the security which I also worry about.
I'm not tasked to do the actual migration, just to look for potential solutions, but I don't know where to look at.
Is it common to have cloud-hosted CI ? If so, are there resources you advise me to read ? Do you have an opinion on how I should do?
Thanks a lot.
https://redd.it/1boysrj
@r_devops
Senior Devops doesn’t know anything. It’s really frustrating
So, I recently started working as a DevOps engineer at an online shopping platform, and boy, let me tell you, it's been a rollercoaster ride already. The interview process was insanely hectic, and there were very few openings available.
Here's the kicker: my senior doesn't seem to know much about what we're actually working with. We're using GCP and Terraform for infrastructure as code (IaC), but they're clueless about even the basics. I'm talking about not knowing how a simple Cloud Build file is created or how it functions.
In our meetings, it feels like we're discussing nonsense most of the time. They're adamant about creating our own Google Cloud logs because apparently, using Logs Explorer is too expensive. but then they stumble when it comes to knowing which machine types are suitable or understanding the differences between them.
It's frustrating because there are countless talented, unemployed freshers out there who could bring so much more to the table. And don't even get me started on CI/CD pipelines – they're a foreign concept to my senior.
Has anyone else experienced something similar? How did you cope with it? Any advice would be greatly appreciated. Thanks for letting me vent.
https://redd.it/1boxx67
@r_devops
What is the best platform for ops automation
For general ops tasks (think database migrations, running hotfixes, etc) what is the best platform?
The only platform i can think of that is dedicated to this kind of stuff is rundeck.
https://redd.it/1bovk40
@r_devops
A new guy might be faking his experience. What should I do?
I’m a DevOps contractor from a consulting company working a staff augmentation project and we just got a new FTE on our DevOps team. It seems one of my consulting teammates thinks that the new FTE is faking some of his past experience.
The FTE says he worked at the same company, at the same time more than 5 years ago, doing the same job, and that they know some of the same people, but our fellow consultant doesn’t remember this FTE. The company we’re talking about is a very large company, but our consultant has reasons to believe the FTE is lying.
On the other hand, this company has an intense interviewing process, and they do employment checks, so if the FTE is here then he passed all of that. The FTE also seems to have some decent skills, seems to do good work, and is very eager to learn.
Personally, I don’t even like my colleagues, they’re assholes. And I think they’re just jealous and hating on this guy & acting like petty hall monitors.
What should we do? Should we report our consultant’s concerns to management, or just let the FTE be?
https://redd.it/1bonrvv
@r_devops
Best place to find Devops Engineer interested in hourly work?
I'm working on an application that uses Django and interacts with a Next.js application using the django-nextjs package (https://github.com/QueraTeam/django-nextjs). There is also a Rust service.
I've been looking for someone to help me create a simple containerized deployment and set up CI pipelines, but I've been having trouble finding someone.
Any ideas of websites or devops communities where I may be able to find someone?
https://redd.it/1bokssb
@r_devops
Who has the nicest docs?
Thinking primarily about charts and containers, who are the people with the best documentation?
Whose Github repo has the best Issue and Pull Request templates?
Looking for inspiration about how to do things the right way. Ease of maintenance and clean formatting. Good ci integration.
https://redd.it/1boh2z3
@r_devops
MONITORING !
For ppl working on monitoring, tell me about your setups, isn’t it painful for those who use nagios to add everything manually for each new server ?
I find prometheus so much flexible with it’s service discovery thing
EKS with prometheus for the service discovery, grafana for visualization, Cadviser for pods, node exporter for OS Metrics and pushgateway for specific development
What do you think, what are all the metrics you think it’s a must to visualize for SR
https://redd.it/1acz0b6
@r_devops
Wrote a post for a workaround for the 'LimitExceeded' error with S3 Object tags in Terraform
Ran into that frustrating 'LimitExceeded' error while tagging S3 objects in Terraform and found a workaround. Thought it might help others here too. I spent far too long googling around and reading GitHub issues, so I thought I would just post about the solution I used, I hope this saves you the time I spent figuring it out :)
Here is the Blog Post
https://redd.it/1adcb5w
@r_devops
Resume Review Been applying for 1 year, can't even get an interview
**Here's my resume**
To put things into perspective, I live in a country with a collapsing economy and practically no jobs. For example, I can't pay to sit for a certificate exam because they've made a $50/month credit card limit. And so, I've sat here at home without a job for a year now.
A family member was very sick for the first 6 months and so I was helping them, but ever since then I've been applying yet unable to get any callbacks. It's always an automated "we will not be proceeding" email. I have no idea how I'll explain my lack of employment if I ever do get an interview.
I don't know what more I can do at this point, I've applied to most DevOps related jobs including software engineers/backend/sysadmins positions but nothing. I'm trying tokeep a positive mindset but everything is looking very bleak as the unemployed gap on my resume keeps growing.
Edit- I have been playing with Kubernetes while applying, and I plan on doing some projects with it. I'll follow up with that by learning Ansible and Gitlab. But self-study and projects can only take you so far...real world experience trumps everything.
https://redd.it/1ad6r91
@r_devops
-backend-config="container_name=$CONTAINER_NAME" -backend-config="resource_group_name=$RESOURCE_GROUP_NAME"
# Run a Terraform apply
- name: Terraform apply
id: apply
env:
ARM_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
ARM_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
run: terraform apply -auto-approve
```
The error suggests an invalid URL request, but as for the root cause of the error, what could I be doing wrong, please? Thanks in advance!
https://redd.it/18woe14
@r_devops
Angry about work tracking process
I've been in my career for 15 years. I'm at the most senior level I can be at for my role within my organization. We had a leadership change a few years back that brought some new processes that I don't like at all.
Entering time in a time sheet by project
Sending a weekly status update with everything you have done
Talking about what you did every day on stand-up
Recording everything you do in a ticket putting hours on it and marking it with a bunch fields for report generation, all data entry and tracking is done by the engineer
It's the last one that bugs me, because I've obviously already provided the info needed to management to track what they need to. However my boss is a self-proclaimed hand-off boss. He had someone create reports off the information we're entering so he can provide updates to management without really being involved with what is going on.
This makes me irrationally angry, and I wonder if I'm being dramatic or if it is actually an awful thing to hoist upon a team. There's more tracking than actual work. He doesn't assign work he just expects it all to get done, he doesn't even care what the work is really.
I thought about leaving but I've got a classic case of golden shackles, there's no way I'd be able to find the same pay with the way the market has changed. I'm thinking about just leaving tech. I also own a real estate business, but it's not big enough to rely soley on. 3 million dollar portfolio with 33 units, I pull in probably 8K extra a month from my business but I put it all back into continue to grow. Eventually, I'll leave tech but my situation currently wont allow for change right this moment.
Really just trying to find a way to accept how much of a cluster this place is or validate my anger. I have much more of an owner mindset, than a manager. I also prefer doing.
Feedback appreciated.
https://redd.it/1bpeccf
@r_devops
Need a FullStack Developer for a freelance project.
I want to have a site/app cloned and result timing reduced by 10-20%. If any capable developer wants to do the work , then DM me for further details.
https://redd.it/1bpb5yd
@r_devops
How are ya'll using ChatGPT or any GenAI tools lately?
Went from ChatGPT Personal to an Enterprise account a few months back and have been advocating for it within my company as a great general purpose tool for pretty much anything. I've seen folks really accelerate their learning on a lot of topics, and just be more productive as a result of it. We've got a few custom GPT's too that a handful of folks use.
We're now exploring Azure OpenAI Studio and Google Vertex AI and Gemini, but again, nothing is in cement yet. ChatGPT is by far the most "mature" (heavy air quotes) tool we're using and even that isn't that popular, I'd say 15% of the company signed up for it, and out of that group I'd say maybe 10% are using it daily.
Some areas I've found it's helping us:
1. Engineers writing way better emails. Really helping as a communications tool.
2. It's half decent at troubleshooting and quick and dirty scripting in bash.
3. It knows enough to be dangerous when it comes to AWS, GCP and Azure. With AWS specifically (90% of our shop is here) it's even helped me get caught up on a lot of new topics.
I can list more but I think these 3 are worth the investment. It's hard to hire these days and I think these tools really help push productivity from the folks who are using them. How about your experiences /r/devops?
https://redd.it/1bp8zv1
@r_devops
I am not a senior and had a first interview with HR for a senior position
I am feeling nervous. I was approached by a recruiter as a Senior DevOps position. I am not a senior and told this the recruiter upfront. The recruiter just talked with the company and they decided to do a HR interview with me.
The interview went well and we will proceed. But I am feeling uncomfortable with the jump since I would be the first and only DevOps for the team. They have a Data Platform with high responsibilities and critical data and would be my responsibility to take care of the environment.
Now I am feeling a very strong imposter syndrome. For example. It is a long time now that I do not touch terraform and they use terraform. Same for Helm. Being the solo DevOps and to take care of the whole environment and also a role to be of very high responsibility sounds too much to me. I am feeling overwhelmed by the responsibilities.
Has anyone had this experience in the past?
https://redd.it/1bp54f8
@r_devops
developer deleted heroku server and the frontend is not loading
so the developer I was working with was probably mad, and he deleted the heroku server without finishing the transition, now some parts of the frontend are not loading... i'm not technical and in the process of finding another developer. Is this easy to fix?
https://redd.it/1bp1t74
@r_devops
how are all of you testing your ansible?
I'm currently undertaking a project to switch our chef to ansible piece by piece, for chef we are leveraging the spec tests which do a decent enough job.
in looking at testing ansible most people seem to mention Molecule but in trying it I can't really seem to find many examples as to how the default verifier "ansible" is actually meant to be written. I've noticed though that many have opted to use testinfra instead.
For local testing in chef we use test kitchen and Molecule seems to emulate that part completely fine.
Curious as to how all of you are managing.
https://redd.it/1boyqdj
@r_devops
Running Tekton Pipelines on Kubernetes at Scale
Running Tekton CI/CD pipelines on Kubernetes/OpenShift at scale: https://piotrminkowski.com/2024/03/27/running-tekton-pipelines-on-kubernetes-at-scale/
https://redd.it/1bowqxj
@r_devops
So Many Hats
Is anyone else in a role/shop where half working code gets thrown over the fence and you have to make it work, all while hearing that the infrastructure is to blame? How does everyone deal with that mental beating? Also, feel free to feel me that I am just doing things wrong and your shop is much better than this 🤣.
https://redd.it/1bopg2o
@r_devops
Self Service Models
Looking through an article about the DevOps Maturity Model, I noticed that the top level gets to self service. What does that look like in your organization? How many people are there?
https://redd.it/1bol1p7
@r_devops
Datadog Question
Hello,
My company started using Datadog recently and we've run into a problem. For context I am a SWE putting on a devops hat for this one. I appreciate any help that anyone can provide.
So, the problem is that we log our requests and the one application has requests around 700 key:value pairs. But, seems to be too large for datadog because they truncate it. Size wise they are about 20kb which is well under the 256KB limit with datadog agent. I can't seem to find any information of how to actually handle this. Datadog just asks why we have such a large JSON object and to respond it is what another team structured their api. Do I need to fight for a change in how large these request objects are or has anyone faced this and have a nice clean solution for logging such "large" objects?
https://docs.datadoghq.com/logs/log\_collection/?tab=hostThis portion below:
The HTTPS API supports logs of sizes up to 1MB. However, for optimal performance, it is recommended that an individual log be no greater than 25K bytes. If you use the Datadog Agent for logging, it is configured to split a log at 256kB (256000 bytes).A log event should not have more than 100 tags, and each tag should not exceed 256 characters for a maximum of 10 million unique tags per day.A log event converted to JSON format should contain less than 256 attributes. Each of those attribute’s keys should be less than 50 characters, nested in less than 10 successive levels, and their respective value should be less than 1024 characters if promoted as a facet.Log events can be submitted with a timestamp that is up to 18h in the past.
Edit: If this sub doesn't seem like the right place for you and have a good idea for another sub to post this in that would also be appreciated.
https://redd.it/1bojkrs
@r_devops
New AWS IAM User - Gruntwork Best Practices
noob question here.
I'm a developer, and have been going through Gruntwork's DevOps best practices ( https://blog.gruntwork.io/an-introduction-to-terraform-f17df9c6d180#3ec6), for a solopreneur journey.
Gruntwork suggests going through their foundations setup, which led me here: https://docs.gruntwork.io/foundations/landing-zone/prerequisites
​
The first thing it tells you to do is:
>A new AWS Account and a user with administrator permissions. (We recommend using an IAM user with admin permissions rather than the root user)
When I go to AWS IAM > Users though, it recommends against creating new users with console access in `IAM Identify Center` (Imgur screenshot: https://imgur.com/a/Fim1NGK). The Gruntwork doc says nothing about this, only about Control Tower. So do I need to be using IAM Users, IAM Identity Center, and Control Tower to achieve these first few steps that Gruntwork is asking me to do?
​
Could someone clarify how you interpret Gruntwork's statement? Or what do you all do for best practices for a multi-account setup.
Basically, I'm starting a fintech and see the value in having separate accounts for `dev`, `staging`, and `production` (or any other separation of concerns that I'll run into when dealing with compliance and such), and I'd like to understand this from the beginning.
​
​
​
https://redd.it/1ad4cqf
@r_devops
2024 Kubernetes Deployment in HomeLab Advice
I'm in the process of setting up a Kubernetes (k8s) cluster in my homelab and am seeking the most efficient and current best practices as of 2024. I'm flexible regarding the distribution.
My current exploration has led me to kubespray and kubeadm. Additionally, I plan to utilize my Synology for storage, integrating it with synology-csi, and aim to manage multiple k8s nodes.
Here are my key requirements:
* Ability to run various applications (such as Paperless, WordPress, etc.)
* Testing scenarios that mirror real-world challenges in my role as a DevOps Engineer.
* The deployment process of the cluster must be idempotent and based on Infrastructure as Code (IaC), enabling regular, code-based deployments.
Is Terraform a viable option for achieving these objectives?
https://redd.it/1adbf1t
@r_devops
Kubernetes Mastery vs. Cloud Provider Expertise for Backend Roles?
Hi everyone,
I am a Software Engineer with yrs of the experience and have to get ready for the interviews.
If you had very limited time to prep for Software Engineer (backend) interviews and had to choose, what do you think holds more weight in the interviews – mastering Kubernetes for container orchestration or practical experience with cloud providers like AWS, Azure, etc.?
what would you prioritize ?
Your guidance would be greatly appreciated. Thank you
https://redd.it/1ad1m6j
@r_devops
Feeling stuck at job
Hi folks, my first post here.
I am currently working as an SRE at Oracle, but really all I do is write python scripts interacting with our APIs, shell scripts, perform deployments using Shepherd (terraform/docker images based), and on call shifts (handle failures).
I want to grow as a devops engineer/SRE. But I really don't see that opportunity in my current team.
What should I do? I am willing to change my job. I hate doing those competitive coding problems which is part of every interview process but I can try, I feel that I'm not really growing in my current job.
https://redd.it/1ad7kjv
@r_devops
Terraform apply fails during GitHub Actions step (HELP WANTED)
'm trying to deploy a simple budget on a subscription in Azure. Using GitHub Actions for CI/CD with OIDC authentication via a service principal.
I'm
However, the following error is outputted on the terraform apply step after the schematic of the planned infrastructure to be deployed is shown:
```
│ Error: checking for presence of existing resource group: resources.GroupsClient#Get: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: error response cannot be parsed: {"<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\"http://www.w3.org/TR/html4/strict.dtd\">\r\n<HTML><HEAD><TITLE>Bad Request</TITLE>\r\n<META HTTP-EQUIV=\"Content-Type\" Content=\"text/html; charset=us-ascii\"></HEAD>\r\n<BODY><h2>Bad Request - Invalid URL</h2>\r\n<hr><p>HTTP Error 400. The request URL is invalid.</p>\r\n</BODY></HTML>\r\n" '\x00' '\x00'} error: invalid character '<' looking for beginning of value
│
│ with azurerm_resource_group.rg,
│ on main.tf line 25, in resource "azurerm_resource_group" "rg":
│ 25: resource "azurerm_resource_group" "rg" {
```
`main.tf` below:
```
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~>3.80.0"
}
}
backend "azurerm" {
key = "budget.tfstate"
use_oidc = true
use_azuread_auth = true
}
}
provider "azurerm" {
skip_provider_registration = true
features {
resource_group {
prevent_deletion_if_contains_resources = true
}
}
}
resource "azurerm_resource_group" "rg" {
name = "budget-rg"
location = var.rg_location
tags = local.common_tags
}
```
### What I've done:
1. Ensured the resource group: `azurerm_resource_group` is created before being referenced by other resources.
2. Ensured there aren't any errors in the config files.
3. Authenticating to Azure is working via OIDC. Exemplified by the `az login` step in my YAML files output:
```
Run azure/login@v1
Running Azure CLI Login.
/usr/bin/az cloud set -n azurecloud
Done setting cloud: "azurecloud"
Federated token details:
issuer - https://token.actions.githubusercontent.com
subject claim - repo:ZimCanIT/zimcanit-az-cloud-resume:ref:refs/heads/main
Attempting Azure CLI login by using OIDC...
Subscription is set successfully.
Azure CLI login succeeds by using OIDC.
```
`push.yml` file triggering run below:
```
name: Push
# trigger pipeline if there is a push targeting the main branch within the az-infra/ subfolder
on:
push:
branches:
- main
paths:
- 'az-infra/budget**'
env:
TF_LOG: DEBUG
permissions:
id-token: write
contents: read
jobs:
deploy-infra:
name: 'Deploy terraform infrastructure'
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: 'az-infra/budget' # keep working tfm files in budget directory
steps:
# Checkout the repository to the GitHub Actions runner
- name: Checkout
uses: actions/checkout@v3
# Install the latest version of Terraform CLI
- name: Setup Terraform
uses: hashicorp/setup-terraform@v2
# Log into Azure with OIDC integration
- name: 'Az CLI login'
uses: azure/login@v1
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
# Run az commands to confirm sub access
- name: 'Run az commands'
run: |
az account show
az group list
# Run Terraform init
- name: Terraform Init
id: init
env:
STORAGE_ACCOUNT: ${{ secrets.STORAGE_ACCOUNT }}
CONTAINER_NAME: ${{ secrets.CONTAINER_NAME }}
RESOURCE_GROUP_NAME: ${{ secrets.RESOURCE_GROUP_NAME }}
ARM_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
ARM_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
ARM_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
run: terraform init -backend-config="storage_account_name=$STORAGE_ACCOUNT"
