Bash Scripting - Revision history

Leo at 23:53, 16 September 2022

2022-09-16T23:53:47Z

← Older revision		Revision as of 23:53, 16 September 2022
Line 1:		Line 1:
	~~<br />~~
	==Useful functions==		==Useful functions==
	Here are some common functions and commands I use in my scripts.		Here are some common functions and commands I use in my scripts.
Line 105:		Line 104:
	echo ${Path/resolv.conf/hostname<nowiki>}</nowiki>		echo ${Path/resolv.conf/hostname<nowiki>}</nowiki>
	}}		}}
	\|}\|}		\|}
			\|}

	==Useful scripting patterns==		==Useful scripting patterns==

Leo at 20:01, 15 June 2022

2022-06-15T20:01:39Z

← Older revision		Revision as of 20:01, 15 June 2022
Line 94:		Line 94:
	\|Get the URL from <a href="...">		\|Get the URL from <a href="...">
	\|<code>''echo '<a href="asdf">'<nowiki/>''<nowiki> \| grep -Eo 'href="[^\"]+"' \| sed 's/href="$.*$"/\1/'</nowiki></code>		\|<code>''echo '<a href="asdf">'<nowiki/>''<nowiki> \| grep -Eo 'href="[^\"]+"' \| sed 's/href="$.*$"/\1/'</nowiki></code>
	\|}		\|-
			\|Replace substring in Bash
			\|
			{\| class="wikitable"
			\|-
			\|Use <code>${variable/search_term/replace_term}</code>. Eg:{{highlight\|lang=bash\|code=
			Path="/etc/resolv.conf"

			# This returns /etc/hostname
			echo ${Path/resolv.conf/hostname<nowiki>}</nowiki>
			}}
			\|}\|}

	==Useful scripting patterns==		==Useful scripting patterns==

Leo at 00:55, 1 December 2021

2021-12-01T00:55:08Z

← Older revision		Revision as of 00:55, 1 December 2021
Line 18:		Line 18:
	max (){		max (){
	echo $(($1>$2 ? $1 : $2))		echo $(($1>$2 ? $1 : $2))
	}		<nowiki>}</nowiki>
	}}		}}
	\|-		\|-
Line 91:		Line 91:
	\|Get all IPv4 addresses on this host		\|Get all IPv4 addresses on this host
	\|<code><nowiki>ip a \| grep -ohE 'inet [0-9.]{7,15}'</nowiki></code>		\|<code><nowiki>ip a \| grep -ohE 'inet [0-9.]{7,15}'</nowiki></code>
			\|-
			\|Get the URL from <a href="...">
			\|<code>''echo '<a href="asdf">'<nowiki/>''<nowiki> \| grep -Eo 'href="[^\"]+"' \| sed 's/href="$.*$"/\1/'</nowiki></code>
	\|}		\|}

Leo: /* Useful functions */

2021-11-30T04:39:21Z

Useful functions

← Older revision		Revision as of 04:39, 30 November 2021
Line 17:		Line 17:
	\|{{highlight\|lang=bash\|code=		\|{{highlight\|lang=bash\|code=
	max (){		max (){
	~~A=$1~~		echo $(($1>$2 ? $1 : $2))
	~~B=$2~~		}
	echo $((A>B ? A : B))
	~~<nowiki>~~}~~</nowiki>~~
	}}		}}
	\|-		\|-
Line 47:		Line 45:
	}}		}}
	\|-		\|-
	\|Extracting the file extension from a ~~string~~		\|Extracting the file name and file extension from a path
	\|{{highlight\|lang=~~terminal~~\|code=		\|{{highlight\|lang=bash\|code=
	~~Filename~~=~~$(basename~~ /etc/resolv.conf)		Path="/etc/resolv.conf"
	~~Ext~~="$~~{Filename~~##*.}"
	Name="${Filename%.*}"		# Get path parts
	~~Name~~=~~`basename~~ $Filename .~~conf`~~		Directory=$(dirname $Path) # /etc
	}}		Filename=$(basename $Path) # resolv.conf
	\|-
	~~\|Extracting the name of a file without the file extension from a filename~~		# Get filename parts
	~~\|<code>''echo~~ "~~house.txt"''<nowiki> \| sed 's/$.$\../\1/'</nowiki></code>~~		Name="${Filename%.*}" # resolv
	\|-		Ext="${Filename##*.}" # conf
	~~\|Extract the directory name from a path~~
	~~\|{{highlight\|lang=terminal\|code=~~
	~~$ dirname /etc/hosts~~
	~~/etc~~
	}}
	\|-
	~~\|Extracting the filename from a path~~
	~~\|{{highlight\|lang=terminal\|code=~~
	~~$ basename /etc/hosts~~
	~~hosts~~
	}}		}}
	\|-		\|-
Line 84:		Line 72:
	}}		}}
	\|-		\|-
	\|Unix timestamp to human readable time		\|Unix timestamp to human readable time.
			You can use either <code>awk</code> with <code>strftime</code>, or with <code>date</code>.
	\|{{highlight\|lang=terminal\|code=		\|{{highlight\|lang=terminal\|code=
	$ echo 1355514777 {{!}} awk '{ print strftime("%c", $1) }'		$ echo 1355514777 {{!}} awk '{ print strftime("%c", $1) }'
Line 104:		Line 93:
	\|}		\|}

	== Useful scripting patterns ==		==Useful scripting patterns==

	===Get a specific column===		===Get a specific column===

Leo at 04:33, 30 November 2021

2021-11-30T04:33:49Z

← Older revision		Revision as of 04:33, 30 November 2021
Line 1:		Line 1:
	<br />		<br />
	==Useful functions ~~and scripting patterns~~==		==Useful functions==
	Here are some common functions and commands I use in my scripts.		Here are some common functions and commands I use in my scripts.
	{\| class="wikitable"		{\| class="wikitable"
	!Function		!Function
	!Code		!Code
			\|-
			\|Use this to ensure the script runs relative to its location.

			This is useful if your script uses relative paths.
			\|{{highlight\|lang=bash\|code=
			#!/bin/bash
			cd "$(dirname "$0")"
			}}
	\|-		\|-
	\|Find the maximum of two numbers. You can implement min by flipping the operands.		\|Find the maximum of two numbers. You can implement min by flipping the operands.
Line 91:		Line 99:
	\|Convert a multi-line list into a comma delimited string		\|Convert a multi-line list into a comma delimited string
	\|<code>''cat list''<nowiki> \| paste -s -d, -</nowiki></code>		\|<code>''cat list''<nowiki> \| paste -s -d, -</nowiki></code>
			\|-
			\|Get all IPv4 addresses on this host
			\|<code><nowiki>ip a \| grep -ohE 'inet [0-9.]{7,15}'</nowiki></code>
	\|}		\|}

	====Get a specific column====		== Useful scripting patterns ==

			===Get a specific column===
	You can use either <code>awk</code> with delimiters specified with <code>-F</code>.		You can use either <code>awk</code> with delimiters specified with <code>-F</code>.
	{{Highlight		{{Highlight
Line 151:		Line 164:
	}}		}}

	====Preserve header while sorting without using a temporary file====		===Preserve header while sorting without using a temporary file===
	You can sort an output on a specific column while preserving the header and without needing a temporary file by using a shim function that reads and prints the first line before passing the rest of the output down the pipe. Define this function:		You can sort an output on a specific column while preserving the header and without needing a temporary file by using a shim function that reads and prints the first line before passing the rest of the output down the pipe. Define this function:
	{{Highlight		{{Highlight
Line 171:		Line 184:
	}}		}}
	Credit to https://unix.stackexchange.com/questions/11856/sort-but-keep-header-line-at-the-top for this neat trick.		Credit to https://unix.stackexchange.com/questions/11856/sort-but-keep-header-line-at-the-top for this neat trick.
	==~~Test~~==		==Testing things in Bash==
	{{code\|test}} is a bash built-in function to add conditionals in your script. {{code\|[}} is an alias of the {{code\|test}} command and are analogous.		{{code\|test}} is a bash built-in function to add conditionals in your script. {{code\|[}} is an alias of the {{code\|test}} command and are analogous.

Line 278:		Line 291:
	\|}		\|}

	===~~Integer~~===		===Integers===

	Integer tests, with {{code\|test}} or inside single brackets.		Integer tests, with {{code\|test}} or inside single brackets.
Line 458:		Line 471:
	;;		;;
	esac		esac
	}}

	~~==Snippets==~~
	~~===Get First IP Address===~~
	~~{{highlight\|lang=terminal\|code=~~
	~~<nowiki>~~
	~~# ip a \| /bin/grep inet \| grep -v inet6 \| tail -n+2 \| head -n 1 \| awk '{print $2}' \| cut -d'/' -f1~~
	~~</nowiki>~~
	}}

	~~===Run Script within Script Directory===~~
	~~For scripts that need to be running on the same directory as the script location, add this line to the top of the script.~~

	~~{{highlight\|lang=bash\|code=~~
	~~#!/bin/bash~~
	~~cd "$(dirname "$0")"~~
	}}		}}

Leo at 04:28, 30 November 2021

2021-11-30T04:28:30Z

Show changes

Leo: /* Get a specific column */

2021-07-14T19:53:58Z

Get a specific column

← Older revision		Revision as of 19:53, 14 July 2021
Line 373:		Line 373:
	three		three
	\| lang = terminal		\| lang = terminal
	}}Or use the bash built-in function read which can take in a list of variables. The last variable contains all remaining columns ~~as well~~. Unwanted columns can be named repeatedly with <code>_</code> (like in golang) and ignored.{{Highlight		}}Or use the bash built-in function <code>read</code> which can take in a list of variables. Variable assignment by piping to <code>read</code> works in zsh but will only work if used within a subshell in bash. Keep this behavior in mind if you choose to use this method of separating out words in a string. For example:{{Highlight
			\| code = ## Basic variable assignment with read
			$ echo one two three four five {{!}} while read _ second _; do echo $second ; done
			two

			## zsh variable assignment works without a subshell/loop
			% echo one two three four five {{!}} read _ second _
			% echo $second
			two
			\| lang = terminal
			}}The last variable contains all remaining columns. Unwanted columns can be named repeatedly with <code>_</code> (like in golang) and ignored.{{Highlight
	\| code = ## Remember, to get only the 3rd column, we need to read the 4th column out.		\| code = ## Remember, to get only the 3rd column, we need to read the 4th column out.
	$ echo one two three four five {{!}} read _ _ third; echo $third		% echo one two three four five {{!}} read _ _ third; echo $third
	three four five		three four five
	$ echo one two three four five {{!}} read _ _ third _ ; echo $third		% echo one two three four five {{!}} read _ _ third _ ; echo $third
	three		three

	## Delimiters specified with IFS		## Delimiters specified with IFS
	$ echo one,two,three,four,five {{!}} IFS=',' read _ _ third _ ; echo $third		% echo one,two,three,four,five {{!}} IFS=',' read _ _ third _ ; echo $third
	three		three
	\| lang = terminal		\| lang = terminal
	}}In terms of speed, be aware that using <code>awk</code> or <code>tr+cut</code> ~~requires~~ a sub-process ~~which may slow down performance~~. If you are ~~going to do a~~ <code>~~while read line~~</code> ~~statement and then parsing out columns within the loop, it might benefit you to parse the columns out by replacing~~ <code>~~while read line~~</code> with <code>~~while~~ read ~~column1 column2 ...~~</code>. For example:		}}In terms of speed, be aware that using <code>awk</code> or <code>tr+cut</code> are actual commands that spawns a sub-process whereas read is a bash built-in function. As a result, it's always faster to use read to parse out data, especially if in a loop. If you are looping and using <code>awk</code> or <code>cut</code>, perhaps consider replacing it with the bash <code>read</code> equivalent. For example:
	{{Highlight		{{Highlight
	\| code = ## Super slow! Invokes awk 2 times per line		\| code = ## Super slow! Invokes awk 2 times per line

Leo: /* Get a specific column */

2021-01-14T22:19:31Z

Get a specific column

← Older revision		Revision as of 22:19, 14 January 2021
Line 363:		Line 363:
	}}		}}

	Or cut by a specific delimiter and select a specific column. This method cannot handle columns separated by more than one space. If you have neat columns, you'll need to pipe through <code>tr -s ' '</code> first.		Or cut by a specific delimiter and select a specific column. This method cannot handle columns separated by more than one space. If you have neat columns, you'll need to pipe through <code>tr -s ' '</code> first which collapses (or squeezes) the given repeating characters down.
	{{Highlight		{{Highlight
	\| code = $ echo one two three four five {{!}} cut -d ' ' -f3		\| code = $ echo one two three four five {{!}} cut -d ' ' -f3

Leo: /* Get a specific column */

2021-01-14T22:18:20Z

Get a specific column

← Older revision		Revision as of 22:18, 14 January 2021
Line 361:		Line 361:
	three		three
	\| lang = terminal		\| lang = terminal
	}}Or cut by a specific delimiter and select a specific column. This method cannot handle columns separated by more than one space. If you have neat columns, you'll need to pipe through <code>tr -s ' '</code> first.		}}
	~~tr -s ' ' {{!}}~~
			Or cut by a specific delimiter and select a specific column. This method cannot handle columns separated by more than one space. If you have neat columns, you'll need to pipe through <code>tr -s ' '</code> first.
	{{Highlight		{{Highlight
	\| code = $ echo one two three four five {{!}} cut -d ' ' -f3		\| code = $ echo one two three four five {{!}} cut -d ' ' -f3
			three
			$ echo "one two three four five" {{!}} tr -s ' ' {{!}} cut -d ' ' -f3
	three		three

Leo: /* Get a specific column */

2021-01-14T22:17:02Z

Get a specific column

← Older revision		Revision as of 22:17, 14 January 2021
Line 361:		Line 361:
	three		three
	\| lang = terminal		\| lang = terminal
	}}Or cut by a specific delimiter and select a specific column:{{Highlight		}}Or cut by a specific delimiter and select a specific column. This method cannot handle columns separated by more than one space. If you have neat columns, you'll need to pipe through <code>tr -s ' '</code> first.
	\| code = $ echo one two three four five ~~{{!}} tr -s ' '~~ {{!}} cut -d ' ' -f3		tr -s ' ' {{!}}
			{{Highlight
			\| code = $ echo one two three four five {{!}} cut -d ' ' -f3
	three		three

	$ echo one,two,three,four,five ~~{{!}} tr -s ','~~ {{!}} cut -d ',' -f3		$ echo one,two,three,four,five {{!}} cut -d ',' -f3
	three		three
	\| lang = terminal		\| lang = terminal