Bash Scripting: Difference between revisions

Latest revision as of 23:53, 16 September 2022

Useful functions

Here are some common functions and commands I use in my scripts.

Function Code

Use this to ensure the script runs relative to its location.

This is useful if your script uses relative paths.

#!/bin/bash
cd "$(dirname "$0")"

Find the maximum of two numbers. You can implement min by flipping the operands.

max (){
        echo $(($1>$2 ? $1 : $2))
}

Similar to the die command in PHP which prints out a message before stopping execution.

Note that this will not work inside a subshell as it will just quit the subshell rather than the script. To get around this, you may need to kill the parent PID instead.

function die {
	echo $@
	exit
}

Indents text piped to the function.

# Supports up to 10 tabs
function Indent (){
	Tabs="\t\t\t\t\t\t\t\t\t\t"
	sed "s/^/${Tabs:0:$((2*$1))}/g"
}

echo "test" | Indent 1
	test
echo "test" | Indent 2
		test

Extracting the file name and file extension from a path

Path="/etc/resolv.conf"

# Get path parts
Directory=$(dirname $Path)  # /etc
Filename=$(basename $Path)  # resolv.conf

# Get filename parts
Name="${Filename%.*}"  # resolv
Ext="${Filename##*.}"  # conf

Skip the first N lines from an output

## -n +N outputs starting on line N.
## Eg. To skip the first 2 lines, we start on line 3:
# tail -n+3

Get the current Unix timestamp

$ date +"%s"
1596137957

Unix timestamp to human readable time.

You can use either awk with strftime, or with date.

$ echo 1355514777 | awk '{ print strftime("%c", $1) }'
Fri 14 Dec 2012 12:52:57 PM MST

$ date +"%c" -d @15996137957
Thu 30 Jul 2020 01:39:17 PM MDT

Get a file's last modified time in Unix timestamp stat -c %Y filename

Convert a multi-line list into a comma delimited string cat list | paste -s -d, -

Get all IPv4 addresses on this host ip a | grep -ohE 'inet [0-9.]{7,15}'

Get the URL from <a href="..."> echo '<a href="asdf">' | grep -Eo 'href="[^\"]+"' | sed 's/href="$.*$"/\1/'

Replace substring in Bash

Use `${variable/search_term/replace_term}`. Eg: Path="/etc/resolv.conf" # This returns /etc/hostname echo ${Path/resolv.conf/hostname}

Useful scripting patterns

Get a specific column

You can use either awk with delimiters specified with -F.

$ echo one two three four five | awk '{print $3}'
three

$ echo one,two,three,four,five | awk -F, '{print $3}'
three

Or cut by a specific delimiter and select a specific column. This method cannot handle columns separated by more than one space. If you have neat columns, you'll need to pipe through tr -s ' ' first which collapses (or squeezes) the given repeating characters down.

$ echo one two three four five | cut -d ' ' -f3
three
$ echo "one    two    three    four     five" | tr -s ' ' | cut -d ' ' -f3
three

$ echo one,two,three,four,five | cut -d ',' -f3
three

Or use the bash built-in function read which can take in a list of variables. Variable assignment by piping to read works in zsh but will only work if used within a subshell in bash. Keep this behavior in mind if you choose to use this method of separating out words in a string. For example:

## Basic variable assignment with read
$ echo one two three four five | while read _ second _; do echo $second ; done
two

## zsh variable assignment works without a subshell/loop
% echo one two three four five | read _ second _
% echo $second 
two

The last variable contains all remaining columns. Unwanted columns can be named repeatedly with _ (like in golang) and ignored.

## Remember, to get only the 3rd column, we need to read the 4th column out.
% echo one two three four five | read _ _ third; echo $third
three four five
% echo one two three four five | read _ _ third _ ; echo $third
three

## Delimiters specified with IFS
% echo one,two,three,four,five | IFS=',' read _ _ third _ ; echo $third
three

In terms of speed, be aware that using awk or tr+cut are actual commands that spawns a sub-process whereas read is a bash built-in function. As a result, it's always faster to use read to parse out data, especially if in a loop. If you are looping and using awk or cut, perhaps consider replacing it with the bash read equivalent. For example:

## Super slow! Invokes awk 2 times per line
$ w | tail -n +3 | while read i ; do
  Username=$(echo "$i" | awk '{print $1}')
  Source=$(echo "$i" | awk '{print $3}')
  echo $Username $Source
done

## Instead do something like this!
$ w | tail -n +3 | while read Username _ Source _ ; do
  echo $Username $Source
done

Preserve header while sorting without using a temporary file

You can sort an output on a specific column while preserving the header and without needing a temporary file by using a shim function that reads and prints the first line before passing the rest of the output down the pipe. Define this function:

# print the header (the first line of input)
# and then run the specified command on the body (the rest of the input)
# use it in a pipeline, e.g. ps | body grep somepattern
body() {
    IFS= read -r header
    printf '%s\n' "$header"
    "$@"
}

Then use this shim body function with the desired sort function. For example:

$ top -b -n 1 | head -n 15 | tail -n +7 \
  | body sort -k 5 -rn

Credit to https://unix.stackexchange.com/questions/11856/sort-but-keep-header-line-at-the-top for this neat trick.

Testing things in Bash

test is a bash built-in function to add conditionals in your script. [ is an alias of the test command and are analogous.

Use the test operators listed below with test or a single bracket test condition like the following example.

$ test -e /etc/hosts && echo "File exists"
File exists

## Note that the white space around [ and ] and also the operator are required.
$ [ -e /etc/hosts ] && echo "File exists"
File exists

$ if [ -e /etc/hosts ] ; then echo "File exists" ; fi
File exists

Operator	Description
`-e`	file exists
`-a`	file exists (deprecated) This is identical in effect to `-e`
`-f`	file is a regular file (not a directory or device file)
`-s`	file is not zero size
`-d`	file is a directory
`-b`	file is a block device. Eg. `/dev/sda`.
`-c`	file is a character device. Eg. `/dev/tty0`.
`-p`	file is a pipe. Eg. `/dev/fd/0`.
`-h`	file is a symbolic link
`-L`	file is a symbolic link
`-S`	file is a socket
`-t`	file (descriptor) is associated with a terminal device This test option may be used to check whether the stdin [ -t 0 ] or stdout [ -t 1 ] in a given script is a terminal.
`-r`	file has read permission (for the user running the test)
`-w`	file has write permission (for the user running the test)
`-x`	file has execute permission (for the user running the test)
`-g`	file or directory has set-group-id (sgid) flag set If a directory has the sgid flag set, then a file created within that directory belongs to the group that owns the directory.
`-u`	file has set-user-id (suid) flag set
`-k`	file or directory has sticky bit set If set on a file, that file will be kept in cache memory, for quicker access. If set on a directory, it restricts write permission. Setting the sticky bit adds a t to the permissions on the file or directory listing. This restricts altering or deleting specific files in that directory to the owner of those files.
`-O`	you are owner of file
`-G`	group-id of file same as yours
`-N`	file modified since it was last read
`f1 -nt f2`	file f1 is newer than f2
`f1 -ot f2`	file f1 is older than f2
`f1 -ef f2`	files f1 and f2 are hard links to the same file
`!`	"not" -- reverses the sense of the tests above (returns true if condition absent).

Integers

Integer tests, with test or inside single brackets.

Eg.

$ test 2 -eq 2 && echo equal

Operator	Description
`-eq`	is equal to
`-ne`	is not equal to
`-gt`	is greater than
`-ge`	is greater than or equal to
`-lt`	is less than
`-le`	is less than or equal to

Integer tests can also be done inside double parentheses. Eg.

$ ((2==2)) && echo equal

Operator	Description
`==`	is equal
`<`	is less than
`<=`	is less than or equal to
`>`	is greater than
`>=`	is greater than or equal to

Strings

Operator	Description
`-z`	string is null, that is, has zero length
`-n`	string is not null.
`=`	is equal to ## Does not work in zsh. Works only in sh and bash. $ if [ "$a" = "$b" ] ; then echo "Equal" fi
`==`	is equal to ## Does not work in zsh. Works only in sh and bash. $ if [ "$a" == "$b" ] ; then echo "Equal" fi This operator inside a double bracket `[[ ]]` construct does pattern matching.
`!=`	is not equal to $ if [ "$a" != "$b" ] ; then echo "Not equal" fi This operator inside a double bracket `[[ ]]` construct does pattern matching.
`=~`	Match using regex $ if [ zone36-ta =~ .ta$ ] ; then echo "matches" fi $ if [ zone36-ta =~ ta ] ; then echo "matches" fi

Command Substitution

Command substitution allows output from a command to be substituted in place of the command itself in a script. Commands can be enclosed within backticks `command` or the POSIX compatible form $(command).

## Eg.
echo "Today is " $(date)
echo "Today is " `date`

Do not confuse $(command) with $((expression)) which is used for arithmetic expansion.

Nested Substitution

If you are trying to do something similar to:

Something=`basename `ls /home/*.txt``

You will get an error since the expansion is ambiguous. Instead, use the $(command) substitution instead:

Something=$(basename $(ls /home/*.txt))

Arithmetic Operations

Do integer arithmetic operations on variables within double brackets. For example:

Value=1
ValuePlusOne=$((Value + 1))
ValueTimes12=$((Value * 12))

The operators that are supported are:

+ - / * %

Bitwise operators also work:

^ (XOR)   | (OR)   & (AND)   << (Shift left)   >> (Shift right)

Control Blocks

Switch

read -p "What do you want to do?: " Option

case "$Option" in
	"K")
		install_new_kernel
		die "Complete!"
		;;
	"H")
		usage
		;;
	*)
		die "Stopping installation."
		;;
esac