Shell ain't a bad place to FP: part 2/N: Functions as Unix Tools
https://www.evalapply.org/posts/shell-aint-a-bad-place-to-fp-part-2-functions-as-unix-tools/index.html#main
https://redd.it/1f3y5g8
@r_bash
Out of curiosity, how many of you use shellcheck regularly? How deeply and how does it fit into your own workflows?
TL;DR - I'm just curious if/how many others actually use shellcheck, to what extent, how you add it to your own workflows, etc. Feel free to share your own experiences, stories, tips and tricks, cool ideas, alternatives, etc related to shellcheck or linting and debugging bash scripts.
First, for those of you who have no idea what I'm talking about: see here
I've recently been updating some of my older bash scripts to address all the issues pointed out by shellcheck (even most of the style warnings). But I've noticed several scripts that I had yoinked from elsewhere off the web also had quite a few issues. Now, I'm not trying to shame anyone (something about people who live in glass houses not throwing rocks comes to mind lol) but it did get me to wondering how many people actually use the tool. I now have my repos setup to use the pre-commit project (the python based one - great overview here) and have been considering adding a shellcheck hook.
For those who also use it, do you just do a quick and dirty error scan? Do you fix all or most nags possible? Do you run it manually? Via git hooks? Something else?
Just curious how others use it mostly and wanted to discuss with people who will probably know wtf I'm talking about 😁
https://redd.it/1f38s1q
@r_bash
Quick question on filetypes
If I want to do something different depending on filetype, can I just
#!/bin/bash
if -f /path/to/file/*.jpg; then
echo "jpg detected."
elif -f /path/to/file/*.png; then
echo "jpg detected."
else
echo "File file does not exist."
fi
Or is there a better way?
https://redd.it/1f2ivti
@r_bash
GitHub - adityaathalye/oxo: A game of traditional 3x3 Noughts and Crosses, in Bash.
https://github.com/adityaathalye/oxo
https://redd.it/1f2clyf
@r_bash
Litany Against Fear script
I recently started learning to code, and while working on some practice bash scripts I decided to write one using the Litany Against Fear from Dune.
I went through a few versions and made several updates.
I started with one that simply echoed the lines into the terminal. Then I made it a while-loop, checking to see if you wanted to repeat it at the end. Lastly I made it interactive, requiring the user to enter the lines correctly in order to exit the while-loop and end the script.
#!/bin/bash
#The Litany Against Fear v2.0
line1="I must not fear"
line2="Fear is the mind killer"
line3="Fear is the little death that brings total annihilation"
line4="I will face my fear"
line5="I will permit it to pass over and through me"
line6="When it has gone past, I will turn the inner eye to see its path"
line7="Where the fear has gone, there will be nothing"
line8="Only I will remain"
fear=1
doubt=8
courage=0
mantra() {
sleep .5
clear
}
clear
echo "Recite The Litany Against Fear" |pv -qL 20
echo "So you may gain courage in the face of doubt" |pv -qL 20
sleep 2
clear
while $fear -ne 0
do
echo "$line1" |pv -qL 20
read fear1
case $fear1 in
$line1) courage=$(($courage + 1))
mantra ;;
) mantra
esac
echo "$line2" |pv -qL 20
read fear2
case $fear2 in
$line2) courage=$(($courage + 1))
mantra ;;
) mantra
esac
echo "$line3" |pv -qL 20
read fear3
case $fear3 in
$line3) courage=$(($courage + 1))
mantra ;;
) mantra
esac
echo "$line4" |pv -qL 20
read fear4
case $fear4 in
$line4) courage=$(($courage + 1))
mantra ;;
) mantra
esac
echo "$line5" |pv -qL 20
read fear5
case $fear5 in
$line5) courage=$(($courage + 1))
mantra ;;
) mantra
esac
echo "$line6" |pv -qL 20
read fear6
case $fear6 in
$line6) courage=$(($courage + 1))
mantra ;;
) mantra
esac
echo "$line7" |pv -qL 20
read fear7
case $fear7 in
$line7) courage=$(($courage + 1))
mantra ;;
) mantra
esac
echo "$line8" |pv -qL 20
read fear8
case $fear8 in
$line8) courage=$(($courage + 1))
mantra ;;
) mantra
esac
if $courage -eq $doubt
then
fear=0
else
courage=0
fi
done
https://redd.it/1f1wfv9
@r_bash
I wrote a puzzle game entirely in bash
https://redd.it/1f1l58l
@r_bash
Ward – a file vault written in bash (github.com/oeo)
just wanted to share ward, a tool i hacked together to encrypt and manage sensitive files in a vault folder. it's written in bash and meant to be cloned and then stored using git or version control.
you don't have to store it that way, it's just why i created it.
what ward does:
* encrypts the files in ./vault/ into a single .gpg file
* checks to see if your files have been tampered with
* generates totp codes if you need them to recover critical accounts.
how to use it:
* clone the repo
* toss your files into the vault directory
* run yarn encrypt.
* now commit your new \[private\] repository or save it somewhere.
that's it. repo link: [https://github.com/oeo/ward](https://github.com/oeo/ward)
feedback welcome, or just let me know if you find it useful.
https://redd.it/1f1fpfb
@r_bash
sed command
I'm learning how to use the sed command. I found the following in a script that I was trying to understand:
sed 's#"node": "#&>=#' -i package.json
The line that this command modifies is:
"node": "20.15.1"
The syntax for sed is supposed to follow:
sed OPTIONS... SCRIPT INPUTFILE...
Does putting the option -i after the script change how the command functions in any meaningful way or is this just non-standard usage?
https://redd.it/1f0vsk9
@r_bash
GitHub - TheKrystalShip/KGSM: A bash cli tool to install/update/manage game servers
https://github.com/TheKrystalShip/KGSM
I've been working on this for the past few months and I'd like to share it with the community. This is my first project in bash, pretty much learned as much as I could along the way and it's at a point where I feel relatively confident about putting it out there for other people to see/hopefully use.
It's a project that came into existence because of my own personal need for something exactly like this (yes I know about the existence of LGSM, nothing but love to that project <3) and I wanted to try and challenge myself to learn how to make decent bash scripts and to learn the internals of the language.
If you're in the market for some light tinkering and you happen to have a spare PC lying around that you can use as a little server, please try out the project and leave some feedback because I'd love to continue working on it with new outside perspectives!
Thank you for your time
https://redd.it/1f0etrn
@r_bash
bash-timer: A Bash mod that adds the exec time of every program, bash function, etc. directly into the $PS1
https://github.com/hopeseekr/BashScripts/tree/trunk/bash-timer
https://redd.it/1f0aq3g
@r_bash
Issues with trying to store a tmp file as a variable.
I'm making something that writes an script that will wrap around a symlink located in /usr/local/bin
Before I was simply using
cat <<-"HEREDOC" >> "$TMPFILE"
content of wrapper script here
HEREDOC
then ask some questions with a for loop that would edit the $TMPFILE with sed -i and as the final step, the symlink in /usr/local/bin gets replaced with the $TMPFILE and the wrapper script is placed in the original place of the symlink.
I've been trying to avoid making a temp file, and instead storing the wrapper script in a variable as it is being made:
tmpscript="$(cat <<-'HEREDOC'
content of wrapper script here
HEREDOC
)
And simply tmpscript$(echo $tmpscript | sed etc etc) to edit it. Which works all nicely.
Now here is where the problems start.
I tried doing:
$SUDOCMD echo "$tmpscript" > "$TARGET"
To the replace the original mv "$TMPFILE" "$TARGET" I was doing before.
$TARGET is the path to the symlink
$SUDOCMD is either sudo or doas depending on what's available
The first issue I had was that the echo "$tmpscript" > "$TARGET" was following the symlink and replacing the actual file that the symlink pointed to, so I fixed that issue by changing it to:
$SUDOCMD rm -f "$TARGET"
$SUDOCMD echo "$tmpscript" > "$TARGET"
For some reason the last step is giving me a permission denied error? but SUDOCMD is being set to doas in my case and it works to remove the $TARGET symlink, why does it fail right after?
https://redd.it/1ezjfuy
@r_bash
what separates a string in bash?
so i didn't want to have to make a completely new thread for this question, but i am getting two completely different answers to the question
what separates a string in bash?
answer 1: a space separates a string
so agdsadgasdgas asdgasdgaegh are two different strings
answer 2: quotes separate a string
"asdgasgsag agadgsadg" "asgdaghhaegh adsga afhaf asdg" are two different strings
so which is it? both? or one or the other?
thank you
https://redd.it/1eyynw1
@r_bash
learning bash
hi i am learning bash (on kali) and i cant figre out what is the error tryid ai but with no luck code:
#!/bin/bash
read -p 'username: ' name
read -sp 'password: ' pass
entered = $1
echo your user name is: $name your password is: $pass inputted number is: $entered
if someone recommend a totrail say to me
https://redd.it/1eyo77n
@r_bash
How do i activate a wireless network?
I have a headless raspberry pi. I messed up some wifi settings by adding a new access point, then ssh connection closed. I rebooted the device hoping it would connect to the new AP but it couldnt connect.
I have access to the root filesystem through sd card. And when i check /etc/NetworkManager/private_networks, i see that configuration is correct but i guess its not automatically connecting. It doesnt connect to the old network as well...
How do i activate specific network from root filesystem on my raspberry pi?
https://redd.it/1eyiw8r
@r_bash
What command do you use for manage for conversion from jpg to pdf
hi, I like to know if there is a tool for get a pdf sheet form a .jpg file.
I use LO for get a pdf file, using a jpg with the size of 1 standard A4 page from LO (Libre Office).
I had qpdf tool but in its man it says that it is a tool for manage pdf.
I have txttopdf too ¿txt to pdf? I don't remember but it is for text.
Regards!
https://redd.it/1f3ejug
@r_bash
Using Lua Instead of Bash For Automation
https://medium.com/gitconnected/using-lua-instead-of-bash-for-automation-196d7e673535?sk=ebcb3cc956499622f293cebedec307fc
https://redd.it/1f33t4e
@r_bash
what is the top command showing me?
so i'm looking at the top command and i'm looking at the man page for the top command and it's weirdly vague
" The top program provides a dynamic real-time view of a running system.
It can display system summary information as well as a list of
processes or threads currently being managed by the Linux kernel. "
it CAN display system summary information?
that's great, but what does it actually display? i understand it can display one thing, but what does it actually display? does it show the processes or applications? is there a difference?
does it show ALL processes? does it show ALL applications? or only the ones above a certain amount of resource consumption?
thank you
https://redd.it/1f2exqw
@r_bash
inotifywait, processing
I have `downloads.txt` in the following format for yt-dlp/youtube-dl to download (a new yt-dlp process is run on each url as it iterates through the file, I can't use its `-a` option to parse the entire file as a batch of urls to download because this file contains the titles of the videos I've set for that url its filename:
<url> <title of video>
<url> <title of video>
<url> <title of video>
...
Script works fine:
while IFS= read -r line || [[ -n "$line" ]]; do
title="$(cut -s -d' ' -f2- <<< "$line")"
url="${line%% *}"
if [[ -n "$title" ]]; then
template="$title - %(uploader).12B (%(upload_date)s) %(id)s.%(ext)s"
else
template="$default_template"
fi
# Ctrl-C should quit script and not just this instance of yt-dlp
# https://mywiki.wooledge.org/SignalTrap#When_is_the_signal_handled.3F
pid=
trap '[[ $pid ]] && kill "$pid"' EXIT
yt-dlp --output "$template" -P "$dir" "$url" &
pid=$!
wait
pid=
done < <( awk '{ sub("^#.*|#.*", "") ; { if ( $0 != "" ) { print $0 }}}' "$file" ) # Ignore blank lines and comments from $file
exit
---------------
Now I want to add `inotifywait -e create,modify "$file"` support so that every time I edit the file, it restarts the loop and runs yt-dlp again. The entire script should stop when `yt-dlp` encounters an error (it will report an error when there isn't a url to download, e.g. if I comment out all the URLs or if the file is cleared) or when I terminate with Ctrl-C.
Any ideas? I'm not interested in other workflows--I need to be able to change what is being downloaded and have yt-dlp download accordingly, i.e. I'm not looking to simply add urls in a queue for yt-dlp to download in sequence.
Much appreciated.
https://redd.it/1f28s3w
@r_bash
Is creating multiple intermediate files a poor practice for writing bash scripts? Is there a better way?
Hi, I often write Bash scripts for various purposes. Here is one use case where I was writing a bash script for creating static HTML in a cron job:
First, download some zip archives from the web and extract them. Then convert the weird file format to csv with CLI tools. Then do some basic data wrangling with various GNU cli tools, JQ, and small amount of python code.
I often tend to write bash with many intermediate files:
clitool1 <args> intermediate_file1.json | sort > intermediate_file3.txt
clitool2 <args> intermediate_file3.txt | sort | uniq > intermediate_file4.txt
python3 process.py intermediate_file4.txt | head -n 100 > intermediate_file5.txt
# and so on
Is it possible to send password into a program through its stdin from Bash without installing any third party software?
Link to the script in question: https://gitlab.com/cy_narrator/lukshelper/-/blob/main/luksCreate.sh?ref_type=heads
Scripts repo: https://gitlab.com/cy_narrator/lukshelper
The script aids in creation of a LUKS encrypted file container that can be used to store sensitive file and transfer in a USB drive or through a cloud storage service like Google drive. Yes, there are many other good third party software like Veracrypt that allows you to do it in a much better way. What I aim is to be able to do something like this without relying on any third party solutions so as to reduce dependencies as much as possible while not compromising on Security. More on it is explained in my [article](https://utsavpoudyal.com.np/posts/Create-Encrypted-File-Container-in-Linux/)
The problem is, I need to enter the LUKS password 3 times. Two times for first creating it (new password + verify password) and again to unlock it to first format with a filesystem. It would be nice if I can make the user input their password through my script, then the script will be the one to supply password to cryptsetup when creating and unlocking the LUKS volume for formatting it with filesystem.
I have hardly written five scripts before. These collection of scripts were written by me with the help of chatGPT so please dont be too mad if it looks god awful. I decided to use bash not because I love it or hate it but because it made the most sense given the situation.
Also please feel free to tell whatever you feel about these scripts. Maby there is a better way of doing what I have done.
https://redd.it/1f1fyoh
@r_bash
what is the "kernel ring buffer"?
so i'm experimenting with the "dmesg" command and it spits out a bunch of text, so i went into the man page and it says
"dmesg is used to examine or control the kernel ring buffer.
The default action is to display all messages from the kernel ring buffer."
what is a kernel ring buffer? what does it do?
thank you
https://redd.it/1f0w577
@r_bash
Insert some words before and after to some files
Hi
I wish to insert some words before and after to some files.
Example
From
2000.doc
2001.doc
2002.doc
----and so on---
2020.doc
To
ABC Company 2000 Net Profit.doc
ABC Company 2001 Net Profit.doc
ABC Company 2002 Net Profit.doc
----and so on---
ABC Company 2020 Net Profit.doc
First question
Can it be done? Is it even doable?
(I have searched the internet and the examples that the search engine displayed have a word or two that are common, that is, appear in all the files. In my case, the files have no common words except for the file extension. The name of my first file starts with 2000.doc and the last one ends with 2020.doc)
Second question
If yes, what are the commands that I should type in a Linux terminal?
Thanks a lot.
https://redd.it/1f0p7uo
@r_bash
Output coloring
Bash Script
When running this command in a script I would like to color the command output.
echo
log_message blue "$(printf '\e[3mUpgrading packages...\e[0m')"
echo
if ! sudo -A apt-get upgrade -y 2>&1 | tee -a "$LOG_FILE"; then
log_message red "Error: Failed to upgrade packages"
return 1
fi
output:
[https://ibb.co/jMTfJpc](https://ibb.co/jMTfJpc)
I have researched a method of outputting the command to a file making the color alterations there and display it. Is there a way to color the white output without exporting and importing the color?
https://redd.it/1f0bb8z
@r_bash
My build_and_run.sh evolution during learning how to C & bash
https://redd.it/1f07x29
@r_bash
Unable to remove a word from the names of files
I have a collection of files whose names are in the following order:
ABC Company 01 Priority Member Account.docx
ABC Company 02 Priority Member Account.docx
ABC Company 03 Priority Member Account.docx
......and so on.......
I wish to remove the words `Priority` and `Account` from the each of the above files such that the results are:
ABC Company 01 Member.docx
ABC Company 02 Member.docx
ABC Company 03 Member.docx
....and so on.....
In a terminal, I typed:
username@hostname:~$ cd test
username@hostname:~/test$ for f in *Priority *; do mv "$f" "${f/Priority}"; done
mv: cannot stat '*Priority': No such file or directory
username@hostname:~/test$
Despite the warning message that `mv: cannot stat '*Priority': No such file or directory` the names of the files were changed to:
ABC Company 01 Member Account.docx
ABC Company 02 Member Account.docx
ABC Company 03 Member Account.docx
......and so on...........
Next, I wish to remove the word `Account` from the names of the files.
In a Linux terminal, I typed
username@hostname:~/test$ for f in * Account; do mv -n -- "$f" "${f//Account }"; done
mv: cannot stat 'Account': No such file or directory
username@hostname:~/test$
Nothing happened. The word `Account` still remained.
Could someone provide me the correct command please?
Thanks.
P.S.: Do you think that the command `for f in *Priority *; do mv "$f" "${f/Priority}"; done` can be improved?
https://redd.it/1ezwtx8
@r_bash
Can a conditional return a capture group?
Hello,
The test sample is
009026405
01009556500
226-356-839
00829029200
008-018-454
009-513-213
00383951900
000147765000
I want to use named capture groups. I want my search pattern to match every line where the number of positions of constituent digits is a multiple of 3. Thus, a line may comprise three or four groups of 3 digits each. If the trailing fourth group exists, it should match.
I thought that a method to achieve the last requirement could be a conditional syntax containing a back-referenced named group so that my pattern would go smth like this:
^(0-9{3})-?(0-9{3})-?(0-9{3})-?(?<LAST>0-9{3})?((?(\k<LAST>)\k<LAST>)$
The conditional fails as invalid syntax or (without \k), has no result or omits the 4-group number despite the named group match. Isn't it possible to back-reference a named group in the conditional?
https://regex101.com/r/owl7Ix/2
https://regex101.com/r/oJdviZ/2
https://redd.it/1eyxwqd
@r_bash
awk delimiter ‘ OR “
I’m writing a bash script that scrapes a site’s HTML for links, but I’m having trouble cleaning up the output.
I’m extracting lines with :// (e.g. http://), and outputting the section that comes after that.curl -s $url | grep ‘://‘ | awk -F ‘://‘ ‘{print $2}’ | uniq
The problem is that some sites use single quotes for certain links and double quotes for other links.
Normally I’d just use Python & Beautiful Soup, but I’m trying to get better with Bash. I’ve been stuck on this for a while, so I really appreciate any advice!
https://redd.it/1eyt150
@r_bash
Can someone please explain this cryptic script?
I'm not able to follow the awk and apt-* commands. I need every piped command explained. Thank you!
```txt
# source: https://github.com/nodejs/docker-node/blob/main/20/bullseye-slim/Dockerfile
apt-mark auto '.*' > /dev/null \
&& find /usr/local -type f -executable -exec ldd '{}' ';' \
| awk '/=>/ { so = $(NF-1); if (index(so, "/usr/local/") == 1) { next }; gsub("^/(usr/)?", "", so); print so }' \
| sort -u \
| xargs -r dpkg-query --search \
| cut -d: -f1 \
| sort -u \
| xargs -r apt-mark manual \
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false
```
https://redd.it/1eyi703
@r_bash
Remote launch bash script with ssh problem
sshpass -p "pass" scp ~/Documents/Scripts/bash2.sh remoteuser@ip:~/Documents/
sshpass -p "pass" ssh remoteuser@ip bash -s < ~/Documents/bash2.sh
exit
There are no problems with scp, but the second command searches bash2 on my computer, but I need it to search on a remote PC.
https://redd.it/1eydjjp
@r_bash
