Sed

Tasks

Simple search / replace

Global search and replace:

$ sed 's/<search>/<replace>/g'

To replace only the first match:

$ sed '0,/<search>/s//<replace>/'

Modifying a file directly

The search and replace above only does things inline but doesn't modify the file. In order to operate on a file directly, use the -i filename option.

$ sed -i "s/search/replace/g" /path/to/file

You can also glob multiple files.

$ sed -i -e "s/search/replace/g" files*.txt

Recursively search and replace

$ grep -lRZ 'search' . | xargs -0 -l sed -i -e 's/search/replace/g'

Remove HTML tags

$ sed -e 's/<[^>]*>//g' <file>

Remove File Extensions

$ sed 's/\(.*\)\..*/\1/'

eg:

$ echo "house.txt" | sed 's/\(.*\)\..*/\1/'  # returns 'house'

Delete Matching Line

$ sed '/someline/d'

Inserting a Line Before/After a Match

Use: 0,/<regex>/s//<replacement text where \0 is the found line>/

## Add a password line before the first title entry in the grub.conf file.
$ sed -i "0,/^title/s//password --encrypted $Password\n\n\0/" $GrubConfig

To append after, just put \0 before your replacement text.

Another example: To add After=network-online.target in a systemd service file:

...
# lightdm takes responsibility for stopping plymouth, so if it fails
# for any reason, make sure plymouth still stops
OnFailure=plymouth-quit.service

[Service]
ExecStart=/usr/sbin/lightdm
Restart=always
...

Running this will append an 'After' dependency to the service file before the service section.

terminal
# sed -i "0,/^\[Service\]/s//After=network-online.target\n\n\0/" /usr/lib/systemd/system/lightdm.service

Grab Specific Section using Regex

To grab a specific section of a string, for instance '3' in 'Server time offset: 3':

$ echo 'Server time offset: 3' | sed -n -e 's/.*: \([0-9]*\)/\1/p'

This can also be done with grep, though it isn't regex:

$ echo 'Server time offset: 3' | /bin/grep -ohE '[0-9]*'

Join Every Other Line

To join every other line, useful for making key:value pairs, use:

'$!N;s/\n/ /'

Removing Newlines

If the output you have is wrapped with newlines indented in by a known amount, such as:

Fri Jul 24 19:21:55: Dispatched 1 Task(s) on Host(s) <node008>, Allocated 1 Slo
                     t(s) on Host(s) <node008>, Effective RES_REQ <select[type
                     == local] order[r15s:pg] >;

You can use sed to replace a newline followed by a specific number of spaces to make this one whole line again:

# cat above | sed ':a;N;$!ba;s/\n                     //g' 
Fri Jul 24 19:21:55: Dispatched 1 Task(s) on Host(s) <node008>, Allocated 1 Slot(s) on Host(s) <node008>, Effective RES_REQ <select[type == local] order[r15s:pg] >;

My understanding of how this works is:

• :a creates a new label 'a'
• N to join next line to the current line
• $! if the last command failed (no more new lines?), branch to label a
• Otherwise, with the next line joined, we search and replace (like usual) by searcing for \n and replacing it with nothing.

Place a key=value pair in its own line

sed -r 's/[[:alnum:]]+=/\n&/g'

Further Reading

• http://www.catonmat.net/blog/sed-one-liners-explained-part-one/