86
Reddit DevOps. #devops Thanks @reddit2telegram and @r_channels
“Application” Registry, not containers?
So internally we have a very important application that builds to a single exe. Presently our lead dev on that project will compile manually, and puts the file into a SharePoint page, where our deployment team can find it and apply it as needed. They keep copies of every single revision the app gets, and there can sometimes be 3-4+ updates in a single day.
So this is fairly crap, so my plan is to build a pipeline from git, using teamCity. We already use this for our web based products, and it’s working nicely.
So the build part, easy done.
I’d prefer not to back a script to push files into SharePoint if I can, so I’m looking for some kind of system (preferably self hosted) where the final output exe can be registered/stored, where previous versions would also remain available.
S3 kinda comes to mind, but I was hoping for something a bit more user-friendly and packable, so we can push things like release notes from commit notes and build logs to be available alongside the exe’s, but not “in” the file.
I love the way container registries (private ones like AWS ECR would be perfect if it did apps too), surely this must be a relatively common situation?
https://redd.it/krmvhz
@r_devops
An opsec realization about what happened at the Capitol
One computer. One single computer is all would take. Some page or intern or secretary who fled the building and their screensaver was slow.
That's all it would take for one of the terrorists that stormed the building to get into the Capitol networks. One single thumb drive to infect the network.
Tell me that you're confident it didn't happen. Tell me that they shouldn't sanitize every single computer in the building.
https://redd.it/ks5psu
@r_devops
JUST NEED SOME ENCOURAGEMENT regarding aws/cloud skills.. ALSO I DONT KNOW WHy this is in all caps.
Hey.
33M here, in NYC. I make about 90-ish as a Sys Admin. Base is 80, 10%-ish bonus, and then free stocks per year. Anywho, I had a roadmap years ago of getting Network+ to my CCNA, to AWS Certs. I actually accomplished all of that, but my job didn't do any AWS tasks, so I actually let them expire.
This is my first decent job where I can actually be cozy and not do much and still get paid decently. Obviously, I got lazy.
I'm a sysadmin but really I don't work that hard. I'm not in charge of much. I've been at my job a couple years but there are 2 coworkers that have been there for 10+. They handle all the big stuff, I handle the day to day stuff like patching, etc.
Anyway, we're moving onto AWS so I guess now that's the kick in the pants I need to really get back onto it. And since I passed the AWS certs in the past, I'm actually in charge of the AWS stuff moving forward.
That being said, I would like to have a cloud oriented type job even if only do S3 stuff at work, and to be completely honest, I want the money that comes along with it as well.
It's just my whole life I did desktop support, or just operations. Nothing back end, no developing, no scripting, no programming, no databases... nothing that important. Unless you count batch files to reboot machines in bunches, lol.
So if I learn AWS again and get the cert again, and know how to manage AWS services... so what? I don't know anything about SQL, databases, programming, containers.. nothing.
It just seems a bit daunting to learn everything from scratch with no experience in any of the specialized stuff, but it seems you need all of that to have a decent paying job with AWS.
Anyone have any general advice to help out with? I've been in IT my whole life so it's not like I'm completely in the dark, I just never needed to learn anything else.
https://redd.it/kru1lj
@r_devops
How to use Helm plugins with Terraform's Helm Provider?
Hello all,
I am currently attempting to use the Helm provider to deploy a chart into AWS EKS. This chart has a couple of secrets that I have had encrypted using AWS KMS and the helm-secrets plugin found here:
https://github.com/jkroepke/helm-secret
When deploying directly through Helm, I can simply wrap the regular deployment with the helm secrets plugin like so:
helm secrets install my-chart . -f .\secrets.my-chart-secrets.yaml
and it successfully installs into my cluster no worries. I've also done a successful terraform apply using the helm provider and without encrypted secrets, but I'd like to store them in Github securely.
I am unsure on how to combined these two strategies so I can deploy using terraform and also use the helm-secrets plugin. To expand, I am unsure on how to use plugins in general with it, and I can't seem to find any examples.
In the documentation:
https://registry.terraform.io/providers/hashicorp/helm/latest/docs
there is a plugins_path argument, but I can't figure out a way to actually execute a plugin during the deployment. The helm_release resource doesn't provide many clues for this either.
Is there currently a way to use a helm plugin? If not, would my best bet then be to run a local exec to decrypt the file with the secrets plugin, pass the file in as a values parameter when doing the deployment, and then clean the workspace? This seems like a messy way of doing it when the secrets plugin has a way of handling the decryption and cleaning as a wrapper.
https://redd.it/krw2ln
@r_devops
create local repository git init not working
it's not creating any files in my folder I'm using Mac is thats a problem
https://redd.it/krww66
@r_devops
Good Zero-Trust Access Solutions on AWS?
I'm looking into a a way for users to get remote access into a private network based on credentials and not IP (i.e. "zero trust"). Preferably I'm looking for something on AWS that's free and open source and I could manage myself.
I'm very much interested in HashiCorp's Boundary software, but it's still in its infancy at the moment. I have heard of Pomerium, Pritunl Zero, and even using Amazon's Application Load Balancer to solve the problem, but I'm not sure how they compare.
Do you have any recommendations?
https://redd.it/krxm6v
@r_devops
What is the main difference between bare metal offerings from Packet vs. AWS, Azure, etc. ?
I'm researching the bare metal market, and with a non-technical background, I'm struggling to understand the key differences between a bare metal offering from a company such as Packet (now owned by Equinix) vs. bare metal offerings from AWS, Google, Azure, and others. How do they compare on performance? I know Packet is known for great automation and support, is that something AWS and others do well too?
https://redd.it/kruopz
@r_devops
Agile Line Name Suggestions
So my department is standing up a new line with the sole focus of cloud contact center solutions. Currently we have two on prem contact center teams: Game of Phones, and Lords of the Ring, I was hoping to keep a similar theme. They asked me (I came up with the previous two 5+ years ago so I’m the name guy now) to come up with a list to help the team out with picking one. They asked to have some reference to cloud in the name and I’d like to keep it themed around pop culture.
I had simple ideas like Cloud Wars or The cloud awakens but can’t think of a way to combine all three... cloud pun, phone pun, pop culture.
https://redd.it/kr086b
@r_devops
Purchasing Processes
Hey All,
I'm in the process of building a tool to streamline remote access to K8s clusters.
We're almost to the finish line and are trying to determine the right price for this SaaS tool.
Ideally, we want engineers with a small project scope to use the tool without having to go through a complicated procurement process or run the purchase decision up the chain of command. When I was a dev working at a large tech company, tool procurement was always such a nightmare, we want to fix that when possible.
I think it would be interesting to compare purchasing practices across different industries. Please reply with the following format:
Industry:
Company size (est.):
Company location (country):
Your Seniority (dev, sr, manager, director, etc.):
Price amount when you ask your boss to purchase something:
Price amount when you ask your boss's boss to purchase something:
Price amount when you ask procurement to purchase something:
​
https://redd.it/kradle
@r_devops
Chrome Extension Internal Tools?
Hi everyone! I'm a recent CS grad and I've been really interested in custom internal Chrome Extensions. I've compiled a list of examples from LinkedIn, and a surprising amount of them are for developer-focused workflows.
I would love to ask the community if you have built Chrome Extensions for devops! And, if so why?
https://redd.it/krb8sn
@r_devops
What's the point of running a dockerized application inside a VM box?
What's the point of running a dockerized application inside a VM box? We're running a dockerized application inside a Linux VM box and I was wondering why not just run it on Windows since the dockerized application is just installing Linux.
https://redd.it/kre57a
@r_devops
Is it illegal to make a Mac virtual machine?
Is it illegal to make a Mac virtual machine? I need to make a VM running Mac so I can test websites on Safari. Is it illegal? If so, what are the alternatives?
https://redd.it/krdbj5
@r_devops
Terraform Cost Estimation 2021 Guide
We wrote a guide to help you decide which Terraform cost estimation provider is the best fit based on different use cases (set up, pricing, accuracy, policy integration): https://scalr.com/blog/terraform-cost-estimation/
https://redd.it/kr1dfs
@r_devops
Prometheus/Alert Manager
Hi,
Is there a way to determine if a previous alert that was open is being fired again in Prometheus/Alert Manager setup? I'm trying to run an automation script for an alert only if the alert was fired for the first time. If the script doesn't resolve the alert, it should not run again and instead send a notification.
​
Thanks in advance.
https://redd.it/kr3jm9
@r_devops
ZAP seems to incorrectly report path traversal vulnerability in Angular app
I'm running OWASP ZAP as part of an automated CI/CD process. I am doing a spider and active scan. The report showed that there is a Path Traversal vulnerability.
This is an Angular 2 site and the javascript application is downloaded and runs in the local web browser so there wouldn't be anything revealed on the server.
\------------------------------------------------Alert Detail
High (Medium) Path Traversal
URL http://localhost:8088/Mydir/login
Method POST
Parameter usr
Attack login
Instances 1
CWE Id 22
WASC Id 33
Source ID 1
\----------------------------------------------------------Request
POST http://localhost:8082/Mydir/login HTTP/1.1 User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0 Accept: application/json, text/plain, */* Accept-Language: en-US Content-Type: application/json Content-Length: 28 Origin: https://localhost Connection: keep-alive Referer: https://localhost/Frontend/ Host: localhost:8088
\----------------------------------------------------------Response
HTTP/1.1 200 Access-Control-Allow-Origin: * Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS Access-Control-Max-Age: 3600 Access-Control-Allow-Headers: * Access-Control-Expose-Headers: xsrf-token Access-Control-Expose-Headers: xsrf-token X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Cache-Control: no-cache, no-store, max-age=0, must-revalidate Pragma: no-cache Expires: 0 X-Frame-Options: DENY Content-Type: application/json Date: Tue, 05 Jan 2021 08:41:50 GMT Keep-Alive: timeout=60 Connection: keep-alive ----------------------------------------------------------------
Please let me know if you need any other information.
https://redd.it/kqxl6r
@r_devops
Can some one help me to authenticate hashicorp vault with self signed ssl cert,key
I have deployed vault with slef signed certificated but I am unable to authenticate by using those through hvac.
can someone help me how we need to pass those to authenticate.
I have followed the same process like https://hvac.readthedocs.io/en/stable/overview.html
Thanks in advance.
https://redd.it/krok0u
@r_devops
Best metric for kubernetes sizing horizontal pod autoscaling?
Looking for the best metric to use for this, looking into trying the collective pod cpu/mem usage % but not sure if theres an easy metric to use
​
Using EKS w ALBs
https://redd.it/kroqh6
@r_devops
Monitoring K8s and infrastructure using Prometheus
I have already installed and set up K8s cluster. Prometheus is installed using prometheus operator. When i open prometheus UI there are lots of metrics and i do not know most of them. I have been searching for some documentation about these metrics but nothing concrete for now.
I need in detail explanation for these metrics or at least some documentation. This will be helpful to write my queries.
https://redd.it/krvnhr
@r_devops
Anyone running their ci/cd server on nomad?
What CI/CD servers are yall using on nomad?
i've got a nomad cluster that i really really enjoyed making, so easy.
Now i'd like to start moving some CI/cd agents on to it. But i'd like first class integration.
I haven't been able to find many solutions besides Waypoint, and gitlab, and we JUST moved to github so hard sell on buying another thing.
So what CI/cd solutions host well on nomad? i mostly just need a script schuedler/checkout handler, nothing fancy.
https://redd.it/krzizt
@r_devops
Jenkins VM with GKE agents on JNLP
Anyone using a Jenkins VM ( currently in GCE ) with GKE agents?
Im trying to configure it, and ive got the connections to GKE working, ( Test Connection is good ),
Im using this image - https://hub.docker.com/r/jenkins/inbound-agent/dockerfile
I CAN make a pod.
So my connection does begin.
It DOES say :Successfully pulled image "jenkins/inbound-agent"
But afterwards it just deletes is and recreates a new one in a loop.
​
If i leave the Command to Run as blank, it loops and fails, if i set a command to something like "top", it sits for longer as an active pod. ( in my case for 300secs just to be able to diagnose ), and if i go to the Jenkins master - Nodes and check that agent, i see:
Created Pod: jenkins-agents/jenkins-slave-xb55m
Waiting for agent to connect (29/300): jenkins-slave-xb55m
Waiting for agent to connect (58/300): jenkins-slave-xb55m
Waiting for agent to connect (87/300): jenkins-slave-xb55m
Waiting for agent to connect (116/300): jenkins-slave-xb55m
Until it runs out and we repeat..
​
Any suggestions?
https://redd.it/krplgy
@r_devops
For GitLab's CI, should deployment server information be stored in the gitlab-ci.yml directly, or is there a way to decouple it from the git itself?
We're setting up CI for the first time with GitLab (self-hosted). We have a single local staging server, but will be deploying to several production servers, and we use docker on both the staging and production.
At the moment we just have the staging server information setup inside of the gitlab-ci.yml, and this is largely duplicated across several projects. But this feels wrong to me?
The first reason is because we're duplicating the production server information across multiple gits, so if we change the server we will have to go and change each one.
And secondly because it seems we're coupling this information to each git project, and it doesn't feel related to them. E.g. we have an api project, and storing the production server information in the yml file in there doesn't feel like it should be there, as it's not really part of the api, but how it's deployed.
In regards to the second point it also feels like that for the container registry as well, e.g. we have the container registry and name setup in the yml file. Am I overthinking that, or should that information be stored elsewhere?
For the production deployment we plan to do a similar thing, except we are going to have it call a script on one manager of the docker swarm, and that will deploy and setup services for the others. Again is this a good way to do it?
My main question is, is there a better way to handle this? So that the production server information isn't stored in the git in the gitlab-ci.yml? And should things like the location and name of the container and container registry be kept in there/the git as well?
https://redd.it/krsk4c
@r_devops
Best sources to prepare for Professional Cloud Architect Certification
In my current workplace I need to pass googles "Professional Cloud Architect Certification". I have some practical knowledge in Google Cloud Platform, but from my experience with AWS certification there are a lot of micro details that could be missed in day to day work. So could you please share cources, books or other sources that helped you preparing to the certification.
https://redd.it/kqzcyf
@r_devops
Want switch to Cloud Computing and Operations side
Hi guys, hope you're all safe and sound. Well, I am working as a software engineer(a/c to my designation) in a big private company and right now I am assigned some websites project in which I should just have to solve some issues or bugs and I am doing it for the last 8 to 9 months. But the problem is I want to switch to the cloud computing side or at least the DevOps part because that's what I want to be in the future. As of now, I didn't get any chance to work some operations part in any of my projects and I really want to be a cloud engineer or DevOps engineer. Please tell me what should I do now? I am getting depressed when I think about it. I cannot afford to switch to any other company as in these pandemic days, it will not be a good idea. Guys do suggest to me how do I take my career path into what I want.
Thanks
https://redd.it/krqat9
@r_devops
Sealed secrets
Does it make sense to seal and store secrets in Git instead of using tools like Hashi Vault?
​
\>> https://youtu.be/xd2QoV6GJlc
https://redd.it/kroosp
@r_devops
Is there an issue to using Hyper-V?
Is there an issue to using Hyper-V? I can't remember what it was, but I am pretty sure using Hyper-V caused issues when trying to make a Linux or Mac virtual machine. Am I correct?
https://redd.it/krd8no
@r_devops
How do you debug a dockerized application without connecting to the vm through ssh?
How do you debug a dockerized application without connecting to the vm through ssh? Do you have any tip? Do people really just rebuild their boxes while adding another log command until they get the info they need?
https://redd.it/krelvh
@r_devops
'The Phoenix Project' and 'The DevOps Handbook'
I currently work in Support and i'm looking to just have a read about DevOps in my spare time (I enjoy reading books :P) and was wondering if these would be good books to read to get started / provide excellent value career wise. I have no development experience and do very little basic automation for my work using python.
The two books I am considering : 'The Phoenix Project' and 'The DevOps Handbook'
Is one better to start with for complete beginners like myself? is prior knowledge required?
Thanks in advance!
https://redd.it/krcs4c
@r_devops
Can you jump straight into DevOps out of college?
A little bit of background...
I’m 28 years old trying to make a career shift. I’m about to finish my 4 year IT degree. Currently I hold all three AWS associate certifications. On my downtime I’ve also picked up Terraform, Ansible (still need some practice), Jenkins, and some Docker as well. I’m also proficient in Python and Node.js. In the past I’ve done some front end dev work for a couple colleagues of mine. My goal is to get a DevOps position in the near future.
Now I understand this position is for individuals who are well into their career and know their shit and I know DevOps is a culture/methodology and not a title.
On my down time I’ve worked a project. In that project I’ve spun up a few EC2 instances using Terraform and configured them using Ansible. The first instance was a Jenkins server to build out a CD pipeline and the other instance was a dockerized python app.
Aside from that how can I get more hands on experience with DevOps?
Also, Is it possible for someone like myself to get straight into DevOps?
https://redd.it/kr1jh3
@r_devops
Looking for advise, on my position.
Hi everyone,
I am looking for a some advise, I joined a company about a year ago as a DevOps engineer. My main job focus is suppose to be on automation, deployment and monitoring.
However, I have found that I am being given more and more responsibilities.
It has gotten to a point now that I am basically managing and guiding the direction of the whole infrastructure for the application.
I feel very uncomfortable doing this since I am inexperienced in some areas and being asked to work on company wide projects like becoming PCI DSS compliant (for who don’t know, is an absolute nightmare, if your not working on a green field project).
I am trying to see all of this as a positive thing that I am being trusted to do this and it’s good for my growth.
However, I can’t help but feel like I am being setup to fail as the business is asking for some requirements that are just not possible with our current resources and the fact that things generally aren’t being managed very well by the people who should be managing this sorts of projects.
Has anyone else felt like this and does anyone have any pointers to help me deal with this affectively?
https://redd.it/kqedyf
@r_devops
Tool for inserting csv for MySQL DB
I want to insert a csv file to MySQL DB after doing some basic validation. LOAD DATA INFILE is candidate, but here I need to perform some basic validations.
Column data types in csv are right data type.
Number of columns are same as db table and csv.
Should I write a custom golang or python script? or I can use open-source tool for this?
I am newbie devops.
Ref -1
Ref -2
https://redd.it/kqxoy7
@r_devops