86
Reddit DevOps. #devops Thanks @reddit2telegram and @r_channels
What the hell is going on at GitLab?
We used to be super into the GitLab ecosystem (we've been a customer for 6 years now), loved the transparency, the speed of releases, and our rep was also awesome. We really liked the product and the company, I myself have submitted merge requests to improve GitLab CE because it was just so good.
Starting this year (maybe slightly before), all of that went out the window. Support tickets are going unanswered for days, our new rep is mostly absent, and we're running into massive breaking bugs with GitLab.com. Has anyone else been noticing a huge decline in GitLab support and software quality? We've run into bugs and GitLab has straight up told us that we'd need to pay their professional services team to work around the bugs they introduced, or just wait until they patch the bugs.
We're at the point where we are in talks with GitHub because at least they respond to our emails.
If you work at GitLab, you probably hate us at this point but we have tickets with unacceptable solutions like this, where we're essentially told "yup, nothing we can do for now, pay our PS team if you want it fixed": https://support.gitlab.com/hc/en-us/requests/418790
https://redd.it/152o4bb
@r_devops
Making it better
This is going to be a bit odd (and long, sorry) I think :)
I’ve been into computers for 40 some years starting as a programmer, then LAN Admin, Unix, Linux, and then into Engineering like Platform and DevOps.
I really like what I do and do it at home too with an extensive homelab. It’s fun! I’m a nut :)
I started a new job recently and I’m kind of stuck.
See, I’ve been in “Make It Better” jobs pretty much my entire career.
With programming it was rewriting code to use structured programming, fixing the code and working towards the one way in, one way out (aka functions) methodology.
With LAN Admin, it was writing batch scripts (using edlin :) ) and creating the ability to log in from any workstation (centralized Windows configurations).
With Unix, I created rough websites back before (as far as I know) doc sites like mediawiki or confluence. Just plain HTML and configuration directory structures for the Unix servers, copying configurations and documenting how things work.
With the move into a Unix/Linux mixture, using something like Lotus Notes for some documentation and other IBM tools for further docs.
A couple of jobs back when I started, the admins were sending 300 emails (documentation) to new admins. I created a wiki site and chivied the team into using it, earning the “did you check the wiki” nickname :) I created a ton of automation with shell scripts (ksh; with a mixed environment, that was the shell that was on all the Unix and Linux servers) and then Ansible (which didn’t work on all the servers so still ksh shell scripts). I created an inventory because the company inventory didn’t know how to handle virtual machines which were starting to take off. I took over the Kubernetes infrastructure and significantly improved it including a ton of documentation that just didn’t exist.
The last job I got so much automation in place that building a complete site from Terraform to Ansible to create a complete installation in 90 minutes (tested and demoed :) ). I created a ton of documentation to help the next person.
That’s the problem. The new job, the main guy writes a ton of scripts. Writes a ton of documentation. He’s great but terribly overworked. But it’s leaving me a little sidelined. Like me, he’s constantly updating the docs and updating or creating configurations and scripts. The boss has finally told him he’s to oversee what we do and to stop doing it yourself. I know it’s hard for him to step back and direct. I’m better at that as I’ve done that, “did you check the wiki” sort of thing before so I’ve been in the ‘mentor’ type slot.
I’m coming to the realization though that this might not be for me. I mean, I am bringing new things to the table. I have a ton of experience. But this guy, who’s great don’t even think I’m knocking his work, is having a hard time letting go even though he’s overwhelmed with work and the expected timelines to get things done.
Anyway, one of the things that percolate up when sitting and reading or learning some new computer thing :)
https://redd.it/151qd1q
@r_devops
Fearless Tinkering with Nix
I wrote an article introducing Nix and its functional devops ecosystem. In it, I discuss the appeal containers + infrastructure-as-code and how Nix can help them overcome limitations. The main focus of the article is on declarative environments, reproducible software distribution, and fearless experimentation.
More here: https://www.heneli.dev/blog/fearless-tinkering-nix
https://redd.it/14yq3se
@r_devops
Software Engineers are taking over all DevOps / Cloud Engineering tasks... what do you see being the future of this field?
Have we come back full circle?
1. Started as a philosophy to reduce silos
2. Became just another job (and hence another silo)
3. Processes were automated, and doing these tasks became easier over time thanks to cloud providers and other tech companies
4. Now Software Engineers are taking over most traditional DevOps / Cloud / SRE tasks
5. \#1 has been achieved as the silos are disappearing, and all that's left are Software Engineers
​
So do you think that DevOps / Cloud / SRE roles are eventually going to be phased out? In most tech companies anyways, the software engineers are responsible for the application end to end (from writing application code, to all the networking, containerization, CI/CD pipelines, automation, monitoring, infrastructure, etc).
https://redd.it/14ymkva
@r_devops
Remote vs Local Development
Will remote development completely take over local?
View Poll
https://redd.it/14ygbvx
@r_devops
Seeking Opinions on Better Stack Alternatives
Hey Reddit community! I recently came across an article discussing the top 5 Better Stack alternatives, which included Pulsetic, Super Monitoring, Hetrix Tools, OffAlerts, and Pingdom. I'm considering switching from Better Stack to one of these alternatives for my business.
Has anyone here had any experience with these monitoring solutions? What are your thoughts on their features, pricing, and overall performance? I'd love to hear your opinions and recommendations to make an informed decision. Thanks in advance!
https://redd.it/14ygcmo
@r_devops
Azure batch vms in unusable state
Hello!I have a batch pool that gets created from a python function inside function app - and I need some advice, as I am newbie and I feel like I cant find this information, or I don't know where to look.My batch account is inside a vnet, and initially my batch pool was created with no network configuration. The nodes ended up in an unusable state everytime because "the batch agent extension provisioning has failed on compute node". I also didnt see the vms being allocated inside any of my subnets, so then I explicitly created it in one of my snets. I did see them allocated in that subnet, but the 'unusable' result was the same. I have no private endpoint for the vms, and the policy of my organization does not allow opening the RDP port publicly, so I cant use RDP to test anything in the vnet. But before configuring the batch to be created in the subnet, I connected to the vm and it seemed like nslookup did find the DNS of the storage account for the batch. The NSGs are already configured and the ports should be fine.Can anyone please give a helping hand?
https://redd.it/14ydyup
@r_devops
nextjs ui app configure at runtime in k8s question
Hi all and thanks for viewing.
I'm using nextjs to build a webapp and looking for advice on making it configurable via environment variables.
I know you can have a .env.development, test, and production file, I also know that you can use the public facing env variables. I wanted to know if I could use server side rendering and proxy our API and then be able to configure those API endpoints to use environment variables for configuration. Basically as it stands now I've tried multiple different configurations but I'm unable to get it to work where at runtime I can override whats in .env.development or .env.production with runtime variables. Right now it seems like it has to happen at build time.
Thanks. Might be an easy question but I can't seem to figure it out.
https://redd.it/14y7x5e
@r_devops
Which configuration management tool should i learn first: Puppet, Ansible, or Chef?
Hey everyone! I'm a beginner to DevOps world therefore i'm looking to dive into the configuration management tools, but I'm not sure which one to start with. I've heard about Puppet, Ansible, and Chef, but I'm having trouble deciding where to focus my efforts.
I would love to hear your thoughts and experiences with these tools. Which one would you recommend for a beginner like me? What are the key differences between them? Are there any particular resources or tutorials you could point me towards? Do you recommend any udemy devops courses provided by KodeCloud? Any advice or insights would be greatly appreciated as I embark on my configuration management journey. Thanks in advance for your help!
https://redd.it/14x0uwp
@r_devops
Running basic web application behind Google OAuth in AWS
Hey all,
There's currently a request to run an inhouse application which is essentially a documentation software.
So they commit changes to the repo the docker image is built and pushed to ECR and then from ECR I was planning on running it using AWS App Runner.
​
The issue is this POC worked fine when the information wasn't sensitive but now we need to implement an auth before access to it.
Probably important to specify we're using a public docker image so editing the application itself to do the any auth doesn't seem like an option to me.
I'm not even sure what is the best way to go about this. I'm seeing nothing for inbuilt Cognito within App Runner.
So my question to you guys if you had a similar scenario what services would you use to restrict access to behind our Google OAuth
https://redd.it/14xpgzn
@r_devops
Packer and Ubuntu 22.04 - Renaming the network interface
I want to change the name of default network interface to eth0 of an Ubuntu 22.04 image, during the packer deployment process. By default, on my testing, it seems to be enp0s3. It could also be in a script afterwards, I don't really care, but I can't figure out how to automate either in packer or in a shell script.
I thought having this section in the user-data file would work, but it doesn't. I just ge a network fail crash error during the packer build.
network:
network:
version: 2
ethernets:
enp0s3:
set-name: eth0
So can anyone tell me how to do this :
1. via packer (user-data? syntax?)
2. Shell script that works for the new way Ubuntu does networking?
Thanks!
https://redd.it/14xmwyl
@r_devops
What is the role of QoS for Pods?
Hey all,
I have seen some companies prefer not to give any resources to their workloads. In these cases, the QoS (Quality of Service) is BestEffort. It means, pod can take upto as many cores of cpu and memory as required. But also I read somewhere that its not a good approach as it might lead to memory leak.
On the other hand some companies give the same number of request and limits to their workloads i.e, QoS - Guaranteed while some organisations prefer giving just the requests not limits i.e, QoS - Burstable.
I wanted to understand your opinions as per your experience, which practise to be used and when? And does QoS makes any major impact on workloads??
https://redd.it/14xjskh
@r_devops
Multirepo trigger in azure devops
After researching i found out only way to do this in azure devops is via azure repos which is annoying and I currently use github. Has anyone maybe a found a way where they were able to do a multirepo trigger or something like this in azure devops using github?
https://redd.it/14xia33
@r_devops
Azure pipeline Variables between tasks
Hey,
Im really struggling with variables between tasks and hope to find some help.
I have one powershell task which parse a yaml file, and extract a variable. This variable im setting like this:
Write-Host("##vsotask.setvariable variable=APPLICATION_NAME;isOutput=true$tempApplicationName")
where $tempApplicationName holds the parsed yaml variable im looking for. I have debugged it to make sure it has value.
In the next task, I have a Kubernetes task, which takes inputs:
- task: Kubernetes@0
condition: succeeded()
displayName: "Waiting for rollout"
inputs:
connectionType: Kubernetes Service Connection
kubernetesServiceEndpoint: ${{parameters.kubernetesServiceEndpoint}}
namespace: ${{parameters.namespace}}
command: rollout
arguments: status deployment/$(APPLICATIONNAME) -n ${{parameters.namespace}}
Resource I been looking at:
https://medium.com/microsoftazure/how-to-pass-variables-in-azure-pipelines-yaml-tasks-5c81c5d31763
${{ variables['APPLICATIONNAME'] }} // which is compile time, so i know it wont work
$(variables'APPLICATION_NAME') // Runtime but never worked either
Any ideas?
https://redd.it/kt246j
@r_devops
Question about pull request CI strategy
I'm working in a company where all CI procedures determined by the DevOps team mostly without asking the developers. We have tests covering almost any place of our applications.
About a week ago one of my pull request branches couldn't pass the CI procedure and I realized that the production branch is being merged to my PR branch (feature) before the CI procedures.
There were no CI issues with the master branch. However, when it gets merged to my PR branch it fails at some CI checks. After digging the errors I found that there is a non-standard configuration included in the production branch and it was conflicting with my PR because what I implement was right (following the RFC's and documentation.)
Now it is my branch that is falling. I asked DevOps to not merge the production branch when running the CI procedures because I want my PR branch to be tested in isolation without changes from the production branch.
However, DevOps declined by saying "This is how it works, otherwise we can't guarantee the production branch stability if we don't merge it to PR branch before CI procedures."
While I see the DevOps argument is valid but I still I'm not convinced to have run CI procedures with the production branch merged to the PR branch.
So what is your CI procedure look like in terms of PR branch testing and do you think what our DevOps team is doing legit?
https://redd.it/ktf7r0
@r_devops
Learn devops for web app
I am currently building a classifieds app using DjangoREST/Next/Postgres which will be run on Hetzner cloud with CloudPanel for some admin in addition to SSH.
Where can I as an indie developer learn enough devops and best practices to safely manage my app and Postgres database?
I am not keen on using Heroku due or Elephant SQL due to their pricing. If my app were to be commercially successful I would hire someone for admin/devops, but in the meantime I will be financially constrained working on this project.
https://redd.it/150axmi
@r_devops
WAF Comparison Project
New open source testing tool allows testing the efficacy of WAF solutions in real-world conditions using millions of web requests.
https://github.com/openappsec/waf-comparison-project
The blog below compared the following popular Cloud WAF solutions: Microsoft Azure, AWS, CloudFlare WAF, F5 NGINX AppProtect, ModSecurity and open-appsec/CloudGuard AppSec.
https://www.openappsec.io/post/best-waf-solutions-in-2023-real-world-comparison
https://redd.it/14zkhbt
@r_devops
Video: Video: What is Backstage + Demo! 🚀
https://www.youtube.com/watch?v=R4qJN6S4qHw
https://redd.it/14ypfxp
@r_devops
simple app for monitoring server log?
Hello,
At the moment I browse my logs from the server manually. I'm thinking about a dedicated open-source app for it, like Logwatch. I want access to a panel or email report with basic information, such as who is trying to connect to my server on which port, attempts to DDOS, etc.
Which app is best for it, or maybe it would be better to write a bash script for it?
I want a simple solution configuration, like Grafana plus Prometheus, which is too complicated for me at the moment.
https://redd.it/14ykiio
@r_devops
AWS/GCP/AZURE Certification?
I currently work in Linux/Bash Automation Devops team. I want to switch in the near future and transition to full Devops roles. Which Cloud certification should I choose for better opportunities?
https://redd.it/14ygy4m
@r_devops
Azure - Where are the batch pool's VMs allocated by default?
I can't find this information, therefore I am asking here. I have a batch account inside a vnet and I am creating a pool from a function app. I can see the VMs are created, but I dont see them in any of my subnets in the vnet. Where are they allocated, do they need private endpoints?
https://redd.it/14yem4w
@r_devops
Do you wish you had become a software engineer instead?
title
View Poll
https://redd.it/14y97we
@r_devops
what resources are good to learn grafana
Im looking to learn grafana and prometheus, can anyone define in simpler terms what these are for and the difference between it since im very confused on how they work. You can also suggest resources that helped u guys which i would greatly appreciate 🙂
https://redd.it/14xwsju
@r_devops
pr-agent - AI-based automated pull-request code reviews (open-source)
CodiumAI's pr-agent provides developers with AI-generated code reviews for pull requests, with a focus on the commits: pr-agent - GitHub
The tool gives developers and repo maintainers information to expedite the pull request approval process such as:
the main theme,
how it follows the repo guidelines,
how it focused,
code suggestions to improve the pull request's integrity.
https://redd.it/14xreqz
@r_devops
Mac M2 PRO VS DELL Latitude
Hi,
I need your advice, which of these two is better for DevOps (especially containerized env and more Ops oriented).
​
Dell Latitude 3530 16GB
Mac M2 PRO 16GB
​
https://redd.it/14xl4n5
@r_devops
Is running devops at a big company really this much more insane than at smaller companies?
Jumped from devops at a shaky Series B startup to what I thought was stability at an 70-year-old food company with 10,000 employees.
I don’t think I’ve ever been so shell-shocked in my career. There are dev teams using conflicting systems, devs using redundant systems, what I could consider sobering incidents every week, tickets for the dumbest shit mixed in with important shit.
This is my third real position, but my first outside of startups.
Is this normal? Seriously considering asking for my old job back as it hasn’t been filled yet.
https://redd.it/14xlzkg
@r_devops
Send AWS Opensearch pdf report to email using AWS SES
Hi Guys,
I am using AWS Opensearch, I have generated reports in pdf. I want to send this pdf report to email using AWS SES. It will be an automatic continuous process after every generate. Does anyone have any idea or guidance?
https://redd.it/14xin2g
@r_devops
Running BOINC (contribute computation resource for science voluntarily) on when my servers have <50% CPU load in spare hours: Has anyone done this? Is this good or bad (e.g. do harm to server stability / not environmental friendly)?
Running BOINC (contribute computation resource for science voluntarily) on when my servers have <50% CPU load in spare hours: Has anyone done this? Is this good or bad (e.g. do harm to server stability / not environmental friendly)?
https://redd.it/kt0kom
@r_devops
Never bend the rules in an effort to prop up a fundamentally flawed system. Instead, follow the rules to the letter in order expose the systemic problems. It's the only way to bring about change.
Just thought I'd toss that one out there, as I am sure some of you are enduring a serious mess right now.
https://redd.it/ktck69
@r_devops
login to github from terminal
anybody know the git input on terminal kind of lost here cause nothing is saving
https://redd.it/kt9ieo
@r_devops