86
Reddit DevOps. #devops Thanks @reddit2telegram and @r_channels
Artifactory: What am I missing here?
I’ve been using Artifactory for a few years at multiple different companies and on different projects and it never ceases to amaze me how useless it is for any non-trivial use case. Pretty much all my teams have used it for are pushes and pulls; I can see that there are other slightly more advanced features, but none that any project I’ve worked on has been willing to pay for. Is there something happening here that makes it more practical than a RAID Linux server hosting a shared filesystem? (Besides maybe the server admin side)
Not saying it’s universally useless, but for this particular type of use it doesn’t seem like the right tool.
https://redd.it/1c8af1q
@r_devops
DevOps & Cloud Architecture Books
Hello DevOps folks,
I kinda find myself to struggle in understanding the big picture when taking on work tasks in my current role (Cloud Engineer - Azure). I feel like that I'm working only on small gears of a huge machine.
My company does consulting for big/medium businesses and I can't really understand how changing some IaC or YAML code provides value to the big picture or environments built with them. I mean, I kinda do, but i struggle to visualise the interconnection of the whole environment.
Based on this, can you recommend 1/2 books that would provide me the "big picture" of DevOps or Cloud Architecture solutions, especially in a consulting point of view? I want so bad to understand from 0 to 100 how to make up a digital DevOps/Cloud solution based on real life scenario but i miss the "knowledge components" (maybe something that is acquired after years of experience?).
Thanks in advance!
https://redd.it/1c86r1z
@r_devops
How do you deal with developers not paying attention to slack
Frustrated. I feel like I'm the only one paying attention to things like alerts, PR requests, pings, or even just conversation. Hours go by in the workday and people aren't responding to my pings or PRs. Builds are broken. I'm the only one reviewing PRs. I know it's a culture thing and an expectation that should be set by management but I don't really have any control or influence there.
Edit: just want to clarify my expectations. We all have work to do and are busy with tasks. I get it. I think it's pretty reasonable to watch for broken builds after a push. Or do respond to messages in a team slack channel within half a day. Or to acknowledge someone asked for you to look at a PR within a few hours (not saying you need to look at it then). If an alert in production fires yes IMO that justifies stopping your current task to investigate. If it's bad alert then fix it or turn it off.
Slack is literally the only way we can communicate with each other for the majority of us WFH. We need to acknowledge each other.
https://redd.it/1c7zbsu
@r_devops
People who have personally migrated from Istio to Cilium, what was your experience like?
We’re pretty heavily committed to Istio, but I can’t deny that.. well almost any of the service mesh solutions are less complex and easier to maintain.
Those who have made the jump to Cilium specifically, what has your experience been and would you do it again? Thanks!
https://redd.it/1c7wxzd
@r_devops
Lots of work opportunities I see lately are looking for full-stack experience, including with application layer
I started my career as a developer and for various reasons, it wasn't for me. I found AWS, cloud infrastructure, CI/CD etc. came to me more naturally and when I finally made the transition, I was happy to be able to focus on that and let the developers take care of the application. At my first company there were a few people who were completely full-stack, but it was normal that platform engineers managed the infrastructure while the developers managed the application. I know that may raise questions about silos but that's another can of worms.
Lately I've been looking for a new job and my company is also looking for a project for me (we're a consultancy). One opportunity that got sent to me today asks for "a background in Enterprise Application Development" and "Proficient understanding of the entire technology stack of the service, from application to infrastructure". A few months ago I inquired to a friend about any work going at his company and he said that me not knowing much about application code might be a problem, as that's just not how they did it there.
The fact is that I really don't know much about application code and there's not much I feel I can offer for it. I get the impression that most of the folks here don't get involved with the application much either. Do you also see this in job advertisements like this lately? Is it ok to just want to keep the focus on infrastructure or is it limiting? I'm in the Netherlands, not sure how different it is between countries.
https://redd.it/1c7umuq
@r_devops
Salary Expectation for 8 yoe in India
Hi Everyone,
May I ask what should be the expected CTC for a Devops Engineer in Bengaluru (India).
Thanks
https://redd.it/1c7s4e0
@r_devops
Running Kong on GKE
Hi, new to Kong, love the concepts. Experimented with Gateway APIs on KIND, worked great. Now need to move it to GKE. There are several things I would like to do that I'm not sure how:
1. How to make the kong-proxy service to use the reserved static ip in GKE? Before I would use the kubernetes.io/ingress.regional-static-ip-name annotation on Ingress, but not sure how to do that with Gateway APIs
2. What is the right way of running multiple replicas of kong? I saw there is replicaCount value in the chart, but there is also a cluster.enabled mode. Should it be enabled as well?
3. Related to 2, how to run those replicas so that they are spread across zones & nodes?
4. I'm installing using kong chart from https://charts.konghq.com and with pretty lean values below. What are other fields you would suggest to specify for a typical production use.
ingressController:
enabled: true
Blue Green and CD on EKS
Hey guys, so at my current company - we are having some tension between product and qa/platform. Currently we are doing manual code freezes so qa can thoroughly test a bunch of things before we push to prod. This is causing tension as product engineering wants things out ASAP. Problem being is we’re getting tons of bugs due to this. We are severely lacking the CD portion of our pipeline, and one of my plans (in agreement with the VP of infra) is to get B/G deploys rolled out using something like Argo rollouts (paired with ArgoCD). This way we can push to “prod” and have QA run as many tests as possible without causing a code freeze. We use automated tests like cypress, but another step will be automating more manual QA processes down the line.
In the meantime is this a good idea, would love to hear pushback. Currently we only have a dev env, and while yes we can do a staging - my exp in past is that things can pass staging but break in prod due to minor discrepancies.
Would love to hear what you guys have been doing, especially revolving b/g and CD. We are GitHub actions shop and deploy with helm.
Also given I have a chance to really architect this however I’d like, let me know how you would do deployments if you had no limitations. My last job we did GitHub actions and argocd as well, but did not have any B/G as there were not frequent app breaking changes.
(Note I am new to this company so I did not set any of the current processes up).
https://redd.it/1c7nnhb
@r_devops
Best way to learn Windows IIS
Hey all! I need to learn Windows IIS and am trying to figure out the best way to do it. Is there a recommended book, tutorial, YouTube series I should be looking at?
Context. I've been hired as a lead software engineer for a brand new department. I need to be able to deploy legacy code (OLD asp) *and* modern django applications on an air gapped system that utilizes Windows Server and Windows IIS.
All my experience is Linux and Mac. I haven't touched any devops stuff in a *really* long time, and back then I did LAMP, mostly small scale and from the developer side.
We are building and supporting internal tools for support staff. We aren't expecting high traffic.
\-------------------------------------
Bonus question/thought. No one around me has any idea of how modern software engineering is supposed to work. The deployment process from the guy I replaced was to write code in production, in the production environment, and hit save (no, I'm not kidding). I'm trying to set up something resembling modern day development practices from scratch.
Is there any benefit to even using Windows Server/IIS? Or is the group I'm working for throwing away money?
https://redd.it/1c7drx5
@r_devops
Service Mesh Preferences- Istio, Linkerd, or Something else?
Ok so we're not a service mesh, nor do we have a stake in either of these products so don't come for us in the comments (though- if you want to talk Gateways, hit us up).
We recently received a guest blog that covered a full guide on traffic management in Istio: https://www.getambassador.io/blog/istio-kubernetes-traffic-management-guide
Now, we've played well with both Istio and Linkerd from an integration + open source perspective, but we're curious...do y'all have one you prefer? Or maybe a third service mesh option you prefer more? Any downsides to either that you've seen?
Give us the best and worst you've had from using either of these. :)
https://redd.it/1c7a2t6
@r_devops
Have we reached a stage of AI/ML models for DevOps related tasks?
Like as simple as providing a solution for certain use cases like provisioning storage capacity as needed or open up n/w ports for a new service that gets spinned up?
I want to see if there are any development out there around data science to deal with day-to-day tasks or even provisioning infrastructure when a use case is provided.
Cheers!
https://redd.it/1c789lc
@r_devops
Anyone needs a Coursera Plus?
I will be inviting you to use Plus for a year (worth $399) on your email (corporate invites) at $39 and obviously you won't be paying me without any proof that you require from and before you are satisfied. If anyone is needy and actually needs it, can dm me. I'll help them!
https://redd.it/1brqoc3
@r_devops
Tekton CI/CD Reusability
Can someone provide a simple example? How to reuse Tekton components?
https://redd.it/1brj8oi
@r_devops
DevOps eng with 6 YOE, planning to learn JS and Node
Hi Everyone,
I'm a Devops Engineer with 6 YOE, I transitioned from the
Ops side of things being a sysad for 2 years.
I'm very proficient in most of the things related to ops but I would like to learn JS and node to be able to understand the development side of things more efficiently. I use python and bash regularly for scripting so I have good scripting skills but lack the knowledge of data structures and algorithms or the core dev skills.
I personally believe a DevOps engineer with good development and operations skills is like a unicorn, I haven't come across anyone with such skills. Mostly people either have operations skills or development skills.
Does learning JS and node makes sense ?
https://redd.it/1brjmgz
@r_devops
Shell prompt ideas
I’m working on customizing my shell prompt. It should contain the following:
- current work directory
- git branch/status (if in a git repository)
- current time (hour:minutes)
- terraform version (if in terraform directory)
- terraform workspace (if in terraform directory)
- python version (if in python directory)
- python venv (if in python directory)
- golang version (if in go directory)
- AWS_PROFILE (if exported)
- Last command exit status
Any other suggestions for useful information to display in the prompt?
Many thanks
https://redd.it/1brgjor
@r_devops
GitOps is matured. Right ?
Hi all,
I was talking to our company DevOps Director and was asking questions on why we don't use GitOps and instead use typical Azure Build and Deploy pipeline.
His answer was GitOps isn't matured and isn't good enough for Standardizing across organization ? (I wasn't given proper reason and don't plan on asking)
I personally don't feel that way. Is there any reason why a team wouldn't opt GitOps and keep it the old way ?
Also, if guys have some good learning material to understand these, it would be helpful.
https://redd.it/1c88thx
@r_devops
Do most companies stick to the big 3 (AWS, Azure, GCP)? How come I never see Digital Ocean or Linode come up?
There are many online tutorials that teach people how to get set up on Digital Ocean/Linode but I noticed that I've never seen people discuss production issues on them. Most people only talk about the big 3. Even job listings only mention the big 3 and never about Digital Ocean.
Do big companies or even small tech companies not consider Digital Ocean and others? Are those renegated to hobby projects or small time customers?
https://redd.it/1c814pa
@r_devops
decided for Flux against Argo now folks want a UI ...
well the title says it. I've worked with both Flux and Argo in the past and I decided for Flux. Now people are asking me for a UI. FML.
I've stumbled upon this: https://flux-subsystem-argo.github.io/website/
has any of you guys used this? It seem well maintained and even has Flux' Daniel Holbach on board.
https://redd.it/1c7xvmc
@r_devops
Just feeling a bit defeated.
Managed to make it to the last interview for the 4th time to see this again: “we decided to go with a candidate with more experience.”
I’m closing in on 6 years of experience. Feels like I’m constantly competing against people with 10+ years nowadays. This wasn’t even a senior role. I know there’s been a tech exodus, but damn.
Anyone else in the same boat?
https://redd.it/1c7ur1a
@r_devops
Jenkins exodus? Is it really a thing?
I recently attended Kubecon CloudNative con, and I can't remember who, but someone mentioned a "Jenkins exodus" occurring in the industry.
Where I work we heavily use Jenkins, with Spinnaker sprinkled on top for cross CI communication and pipeline orchestration mainly.
Seeing all the "cool" tools out there, opensource or not, makes me think we need to evolve for the future. For example, Argo workflows, Tekton, Harness, Octopus Deploy, Gitlab, Azure DevOps, Github actions etc.
We have a mix of legacy and cloud native microservice products so any migration/change would be taken slowly.
I'm interested in what people are using but more importantly what their experience is like.
In the Jetbrains State of Developer Ecosystem report 2023 (https://blog.jetbrains.com/teamcity/2023/07/best-ci-tools/#the-best-ci-cd-tools-in-devops-according-to-our-state-of-developer-ecosystem-report), they polled over 26k developers worldwide and Jenkins, Github Actions and Gitlab CI were top of the pile when it comes to which tools are regularly used, but it doesn't give any context around whether the developers are happy with them!
Be great to hear what you're using, and what your experience is like! TIA
​
https://redd.it/1c7tvg4
@r_devops
How can we update docker containers on edge device with no Internet or should we simply get rid of docker?
We need to deploy docker container on edge devices which wont be having Internet. These devices occasionally connect to a network and one of the device (lets call it H) on the network will have internet access. So, I want to know how we can update docker containers in such scenario. I imagine following two approaches:
1. Create tar of image. Copy it to edge device (say over USB) and then update the image on the edge device.
2. Create local registry on device H. Pull the updated image from remote registry to local registry on device H. Make edge device pull only updated layers from this local registry on H.
I feel option 2 is good given it only moves updated layers between devices making update size small, while tar contains all layers resulting in tar of size 300 MB. So, option 2 was good option till we thought device H will be x86 Windows device. But now we are told that it can be Android or iPad companion device for edge device. We cannot run docker registry on Android or iPad right? So what solution we have remained with for updating docker container on edge device?
We did various docker related POCs. But now after knowing the fact that device H can be android or iPad device, we may have to get rid of docker completely and deploy apps say through other non container approaches say snap etc.
Should we let edge device access Internet through device H say through tethering? We don't want Internet on edge device for security reason, but then should we restrict the Internet access on edge to only servers hosting docker registry? Or there can be any better solution without requiring Internet on edge device at all?
https://redd.it/1c7sb25
@r_devops
Seeking Advice from Respected Experienced DevOps Engineers.
I’m a level 1 seller on Fiverr, offering Linux system and server administration services. To be honest, I haven't completed any course fully yet; I've only done about 30 to 50 percent of each course I started, mainly due to procrastination. However, since I'm good at problem-solving and using Google searches, I accept orders from buyers for projects that I have never worked on and have no idea how to handle.
I’m very interested in DevOps, as I believe that to apply for a job in this field, you must have skills in CI/CD, IaC, etc., and I’ve noticed many Linux sysadmins are moving into DevOps. I've realized I need to learn things properly. I’m currently about 40% through the RHCSA and my goal is to complete it without leaving it incomplete. After that, I'm considering pursuing Docker and Kubernetes as they are critical for DevOps roles. I believe good Linux skills are a prerequisite for DevOps.
I'm also practicing networking fundamentals on EVE-NG. I would like to know if I’m on the right path now. Hearing about your success story, the path you followed, and any recommended courses would be much appreciated. Thank you so much!"
This revised message reflects your updated goals and emphasizes your commitment to completing the RHCSA before moving on to Docker and Kubernetes. It’s great to see that you're setting specific objectives to guide your learning path in DevOps.
https://redd.it/1c7qrsi
@r_devops
Safe terraform apply in CI
I know that some shops do terraform apply as part of their CI.
How do ensure that your CI doesn't do something horrific in production? In terraform plan we trust?
Do you use GitHub Actions? What has been your experience?
https://redd.it/1c7ismb
@r_devops
Resume/CV Review
https://imgur.com/a/fdFWyDk
[Reposting because wrong link\]
Hello all, this is my resume (version 8). I have been approached by recruiters and applied to a lot of positions. However no one has come back with an interview for me, really unsure what it is. I am a DevOps Engineer of 3 years really hoping someone can shed some light weather I am trolling with my resume or improvements. Let me know. Many thanks.
https://redd.it/1c7elez
@r_devops
How is Your Company's Use of AWS Evolving?
Are your companies increasing their use of AWS services, maintaining their current level of usage, or are there instances where projects are being moved back on-premises?
I'm interested in understanding the reasons behind these decisions as well. Whether it's due to cost, security concerns, performance issues, or any other factors....
If you're comfortable sharing, I'd appreciate if you could also mention the industry your company operates in and the scale of your AWS usage. :-)
https://redd.it/1c75bpv
@r_devops
Cloud Native AI - The buzzword
I write monthly newsletter on DevOps and Cloud Native things! Explore Cloud Native AI in recent issue https://open.substack.com/pub/thatdevopsguy/p/cloud-native-ai-is-the-new-buzzword?r=2lyl0&utmcampaign=post&utmmedium=email
https://redd.it/1bru0v1
@r_devops
Subscribing to vendors' status pages
I recently found out that you can subscribe to vendors' status pages and be notified whenever something bad happens on their end. This is really useful! I wrote a short blog post about it that explains how to do that:
https://www.merlinn.co/post/get-popular-tool-incident-updates-in-slack
https://redd.it/1brpxeh
@r_devops
Is our approach good or do we need to adjust?
Need your thoughts on our CICD approach which I found to be already there in our project when I joined. My domain is test automation so I am focusing on driving the automation of different test levels. Please forgive me when I can not explain everything in detail regarding DevOps tools. I am just interested in having a good CICD approach to best as possible implement my vision of automation.
What are we doing? We have about 10-15+ micro services each with its own repository and pipeline. Two teams are working in feature branches. On feature branches unit and component tests are executed as well as additional code scans (fortify, dependency scans, …). How? Every feature branch automatically gets his own (temporary) environment where the service is deployed and ZAP (DAST) scanned.
Then the code is merged via pull requests into „develop“ branch of each service. Now this happens: The pipeline is executed again with all the tests but this time a deployment of the service happens on a permanent environment where services can talk to each other. In addition now integration tests are executed checking the services end to end and between specific services. Again sorry for not being able to explain technically how in details this is implemented.
In addition we have a „uat“ environment and - you guess it - the code is coming from the „uat“ branches. This environment is expected to be more stable - only code which shall go to production later is merged to uat branch and the integration tests validate that the code in uat branch (of each service!!) is working fine with the other code in uat branches of other services.
In general we are talking here about an environment per branch similar to GitLab flow. There is no overhead in environment configuration. The code is „just“ merged from environment to environment. Each service has this set of branches and services of all develop and uat branches (and prod as well naturally) can communicate to each other.
We never did a production rollout yet because the project just started a couple of months ago. What I already can say is that we would need to merge the code to „master“ branch via PR which again calls the pipeline and rolls the software out to production.
Due to the early project stage and because it is hard to find any external references to this approach I am not sure if this approach will work fine for us when it comes to additional features being added later with new services or services which need to be updated, hotfixes needed, etc.
So far so good but will we later end in a merge hell when it comes to merging the code to higher environments (uat and prod)?
Please let me know what you think! Thanks :)
​
​
https://redd.it/1brn6vz
@r_devops
Should i take the offer fellas?
Hey, I've had a thread few weeks ago "DevOps is hard". Since I know how to work with Helm, Kubernetes is not that hard anymore :)
Few weeks later I have a offer for lvl 2 Tech support. Requirments are basic UNIX/Linux understanding, troubleshooting skills, basic knowledge of shell scripting languages, experience with some SQL-like, JIRA. Its an entry-level position with 24/7 support schedule.
My question is should I take the offer and keep grinding or stay in my current job(it's in different industry) get paid more and keep learning until I'm ready for Junior DevOps position. Do you think such an experience is helpful for someone who is pursuing career in the field?
https://redd.it/1brjvtx
@r_devops
How would you devops a minecraft server?
Hi All, Im a software dev trying to get better at devops. Google cloud has a bunch of free credits for 90 days so why not setup a modded minecraft server from scratch
My current setup is a pretty basic terraform where
1. 2 buckets get provisioned (server_files and world_backups)
2. A script checks if the buckets are empty and if they are then local directories are zipped and uploaded to the buckets
3. Compute instance is provisioned and the startup script grabs the latest files from the buckets and runs the server
This works OK but there is probably a better way to do it, I also want to add some more functionality like automatic rolling world dir backups that can be invoked manually if needed.
My first thought would be to add scripts to the server that are invoked with cron and add a lightweight api to allow manual execution or just ssh in when i want to. But atm, updating the server requires completely destroying and recreating it which takes ages and destroys the active world dir. So maybe put that stuff in a docker container so i can rebuild it easily on the server, or maybe do everything with cloud functions that ssh in and run commands
Essentially, it seems like there are a million ways to do this and each has downsides. Looking for some input
cheers
​
https://redd.it/1brdynb
@r_devops