Bash Scripting

Useful functions

Here are some common functions and commands I use in my scripts.

Function	Code
Use this to ensure the script runs relative to its location. This is useful if your script uses relative paths.	#!/bin/bash cd "$(dirname "$0")"
Find the maximum of two numbers. You can implement min by flipping the operands.	max (){ echo $(($1>$2 ? $1 : $2)) }
Similar to the die command in PHP which prints out a message before stopping execution. Note that this will not work inside a subshell as it will just quit the subshell rather than the script. To get around this, you may need to kill the parent PID instead.	function die { echo $@ exit }
Indents text piped to the function.	# Supports up to 10 tabs function Indent (){ Tabs="\t\t\t\t\t\t\t\t\t\t" sed "s/^/${Tabs:0:$((2*$1))}/g" } echo "test" | Indent 1 test echo "test" | Indent 2 test
Extracting the file name and file extension from a path	Path="/etc/resolv.conf" # Get path parts Directory=$(dirname $Path) # /etc Filename=$(basename $Path) # resolv.conf # Get filename parts Name="${Filename%.*}" # resolv Ext="${Filename##*.}" # conf
Skip the first N lines from an output	## -n +N outputs starting on line N. ## Eg. To skip the first 2 lines, we start on line 3: # tail -n+3
Get the current Unix timestamp	$ date +"%s" 1596137957
Unix timestamp to human readable time. You can use either awk with strftime, or with date.	$ echo 1355514777 | awk '{ print strftime("%c", $1) }' Fri 14 Dec 2012 12:52:57 PM MST $ date +"%c" -d @15996137957 Thu 30 Jul 2020 01:39:17 PM MDT
Get a file's last modified time in Unix timestamp	stat -c %Y filename
Convert a multi-line list into a comma delimited string	cat list | paste -s -d, -
Get all IPv4 addresses on this host	ip a | grep -ohE 'inet [0-9.]{7,15}'
Get the URL from <a href="...">	echo '<a href="asdf">' | grep -Eo 'href="[^\"]+"' | sed 's/href="\(.*\)"/\1/'
Replace substring in Bash	Use ${variable/search_term/replace_term}. Eg: Path="/etc/resolv.conf" # This returns /etc/hostname echo ${Path/resolv.conf/hostname} |} Useful scripting patterns Get a specific column You can use either awk with delimiters specified with -F. $ echo one two three four five | awk '{print $3}' three $ echo one,two,three,four,five | awk -F, '{print $3}' three Or cut by a specific delimiter and select a specific column. This method cannot handle columns separated by more than one space. If you have neat columns, you'll need to pipe through tr -s ' ' first which collapses (or squeezes) the given repeating characters down. $ echo one two three four five | cut -d ' ' -f3 three $ echo "one two three four five" | tr -s ' ' | cut -d ' ' -f3 three $ echo one,two,three,four,five | cut -d ',' -f3 three Or use the bash built-in function read which can take in a list of variables. Variable assignment by piping to read works in zsh but will only work if used within a subshell in bash. Keep this behavior in mind if you choose to use this method of separating out words in a string. For example: ## Basic variable assignment with read $ echo one two three four five | while read _ second _; do echo $second ; done two ## zsh variable assignment works without a subshell/loop % echo one two three four five | read _ second _ % echo $second two The last variable contains all remaining columns. Unwanted columns can be named repeatedly with _ (like in golang) and ignored. ## Remember, to get only the 3rd column, we need to read the 4th column out. % echo one two three four five | read _ _ third; echo $third three four five % echo one two three four five | read _ _ third _ ; echo $third three ## Delimiters specified with IFS % echo one,two,three,four,five | IFS=',' read _ _ third _ ; echo $third three In terms of speed, be aware that using awk or tr+cut are actual commands that spawns a sub-process whereas read is a bash built-in function. As a result, it's always faster to use read to parse out data, especially if in a loop. If you are looping and using awk or cut, perhaps consider replacing it with the bash read equivalent. For example: ## Super slow! Invokes awk 2 times per line $ w | tail -n +3 | while read i ; do Username=$(echo "$i" | awk '{print $1}') Source=$(echo "$i" | awk '{print $3}') echo $Username $Source done ## Instead do something like this! $ w | tail -n +3 | while read Username _ Source _ ; do echo $Username $Source done Preserve header while sorting without using a temporary file You can sort an output on a specific column while preserving the header and without needing a temporary file by using a shim function that reads and prints the first line before passing the rest of the output down the pipe. Define this function: # print the header (the first line of input) # and then run the specified command on the body (the rest of the input) # use it in a pipeline, e.g. ps | body grep somepattern body() { IFS= read -r header printf '%s\n' "$header" "$@" } Then use this shim body function with the desired sort function. For example: $ top -b -n 1 | head -n 15 | tail -n +7 \ | body sort -k 5 -rn Credit to https://unix.stackexchange.com/questions/11856/sort-but-keep-header-line-at-the-top for this neat trick. Testing things in Bash test is a bash built-in function to add conditionals in your script. [ is an alias of the test command and are analogous. Use the test operators listed below with test or a single bracket test condition like the following example. $ test -e /etc/hosts && echo "File exists" File exists ## Note that the white space around [ and ] and also the operator are required. $ [ -e /etc/hosts ] && echo "File exists" File exists $ if [ -e /etc/hosts ] ; then echo "File exists" ; fi File exists Operator Description -e file exists -a file exists (deprecated) This is identical in effect to -e -f file is a regular file (not a directory or device file) -s file is not zero size -d file is a directory -b file is a block device. Eg. /dev/sda. -c file is a character device. Eg. /dev/tty0. -p file is a pipe. Eg. /dev/fd/0. -h file is a symbolic link -L file is a symbolic link -S file is a socket -t file (descriptor) is associated with a terminal device This test option may be used to check whether the stdin [ -t 0 ] or stdout [ -t 1 ] in a given script is a terminal. -r file has read permission (for the user running the test) -w file has write permission (for the user running the test) -x file has execute permission (for the user running the test) -g file or directory has set-group-id (sgid) flag set If a directory has the sgid flag set, then a file created within that directory belongs to the group that owns the directory. -u file has set-user-id (suid) flag set -k file or directory has sticky bit set If set on a file, that file will be kept in cache memory, for quicker access. If set on a directory, it restricts write permission. Setting the sticky bit adds a t to the permissions on the file or directory listing. This restricts altering or deleting specific files in that directory to the owner of those files. -O you are owner of file -G group-id of file same as yours -N file modified since it was last read f1 -nt f2 file f1 is newer than f2 f1 -ot f2 file f1 is older than f2 f1 -ef f2 files f1 and f2 are hard links to the same file ! "not" -- reverses the sense of the tests above (returns true if condition absent). Integers Integer tests, with test or inside single brackets. Eg. $ test 2 -eq 2 && echo equal Operator Description -eq is equal to -ne is not equal to -gt is greater than -ge is greater than or equal to -lt is less than -le is less than or equal to Integer tests can also be done inside double parentheses. Eg. $ ((2==2)) && echo equal Operator Description == is equal < is less than <= is less than or equal to > is greater than >= is greater than or equal to Strings Operator Description -z string is null, that is, has zero length -n string is not null. = is equal to ## Does not work in zsh. Works only in sh and bash. $ if [ "$a" = "$b" ] ; then echo "Equal" fi == is equal to ## Does not work in zsh. Works only in sh and bash. $ if [ "$a" == "$b" ] ; then echo "Equal" fi This operator inside a double bracket [[ ]] construct does pattern matching. != is not equal to $ if [ "$a" != "$b" ] ; then echo "Not equal" fi This operator inside a double bracket [[ ]] construct does pattern matching. =~ Match using regex $ if [ zone36-ta =~ .*ta$ ] ; then echo "matches" fi $ if [ zone36-ta =~ *ta ] ; then echo "matches" fi Command Substitution Command substitution allows output from a command to be substituted in place of the command itself in a script. Commands can be enclosed within backticks `command` or the POSIX compatible form $(command). ## Eg. echo "Today is " $(date) echo "Today is " `date` Do not confuse $(command) with $((expression)) which is used for arithmetic expansion. Nested Substitution If you are trying to do something similar to: Something=`basename `ls /home/*.txt`` You will get an error since the expansion is ambiguous. Instead, use the $(command) substitution instead: Something=$(basename $(ls /home/*.txt)) Arithmetic Operations Do integer arithmetic operations on variables within double brackets. For example: Value=1 ValuePlusOne=$((Value + 1)) ValueTimes12=$((Value * 12)) The operators that are supported are: + - / * % Bitwise operators also work: ^ (XOR) | (OR) & (AND) << (Shift left) >> (Shift right) Control Blocks Switch read -p "What do you want to do?: " Option case "$Option" in "K") install_new_kernel die "Complete!" ;; "H") usage ;; *) die "Stopping installation." ;; esac See Also https://www.tldp.org/LDP/abs/html/comparison-ops.html http://sipb.mit.edu/doc/safe-shell/ http://mywiki.wooledge.org/BashFAQ https://github.com/dylanaraps/pure-sh-bible