Sed - Revision history

Leo: Leo moved page Sed Tips & Tricks to Sed: Generic information on sed

2021-12-30T21:21:03Z

Leo moved page Sed Tips & Tricks to Sed: Generic information on sed

← Older revision		Revision as of 21:21, 30 December 2021
(No difference)

Leo at 16:16, 9 July 2021

2021-07-09T16:16:02Z

← Older revision		Revision as of 16:16, 9 July 2021
Line 1:		Line 1:
			== Tasks ==

	==~~Modifying a File Directly~~==		===Simple search / replace===
			Global search and replace:
	~~Using the above tips, you can modify a file directly by using the {{code\|-i}} parameter, with the filename as the second argument~~:
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	$ sed ~~-i "~~s/search/replace/g~~" /path/to/file~~		$ sed 's/<search>/<replace>/g'
	}}		}}

	~~You can also glob multiple files.~~		To replace only the first match:
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	$ sed ~~-i -e "~~s/~~search~~/replace/~~g" files*.txt~~		$ sed '0,/<search>/s//<replace>/'
	}}		}}

	==~~Simple search / replace~~==		==== Modifying a file directly ====
	~~Global~~ search and replace:		The search and replace above only does things inline but doesn't modify the file. In order to operate on a file directly, use the <code>-i filename</code> option.
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	$ sed 's/<search>/<replace>/g'		$ sed -i "s/search/replace/g" /path/to/file
	}}		}}

	~~To replace only the first match:~~		You can also glob multiple files.
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	$ sed ~~'0,~~/<search~~>/s/~~/<replace>/'		$ sed -i -e "s/search/replace/g" files*.txt
	}}		}}

	==Remove HTML tags==		====Recursively search and replace====
			<code>$ grep -lRZ 'search' . \| xargs -0 -l sed -i -e 's/search/replace/g'</code>

			===Remove HTML tags===

	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 29:		Line 32:
	}}		}}

	==Remove File Extensions==		===Remove File Extensions===

	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 40:		Line 43:
	}}		}}

	==Delete Matching Line==		===Delete Matching Line===

	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 46:		Line 49:
	}}		}}

	==Inserting a Line Before/After a Match==		===Inserting a Line Before/After a Match===

	Use: {{code\|0,/<regex>/s//<replacement text where \0 is the found line>/}}		Use: {{code\|0,/<regex>/s//<replacement text where \0 is the found line>/}}
Line 75:		Line 78:
	}}		}}

	==~~Grap~~ Specific Section using Regex==		===Grab Specific Section using Regex===
	To grab a specific section of a string, for instance '3' in 'Server time offset: 3':		To grab a specific section of a string, for instance '3' in 'Server time offset: 3':

Line 87:		Line 90:
	}}		}}

	==Join Every Other Line==		===Join Every Other Line===
	To join every other line, useful for making key:value pairs, use:		To join every other line, useful for making key:value pairs, use:
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 93:		Line 96:
	}}		}}

	==Removing Newlines==		===Removing Newlines===
	If the output you have is wrapped with newlines indented in by a known amount, such as:		If the output you have is wrapped with newlines indented in by a known amount, such as:
	{{Highlight		{{Highlight
Line 113:		Line 116:
	*Otherwise, with the next line joined, we search and replace (like usual) by searcing for <code>\n</code> and replacing it with nothing.		*Otherwise, with the next line joined, we search and replace (like usual) by searcing for <code>\n</code> and replacing it with nothing.

	=== Place a key=value pair in its own line ===		===Place a key=value pair in its own line===
	{{Highlight		{{Highlight
	\| code = sed -r 's/[[:alnum:]]+=/\n&/g'		\| code = sed -r 's/[[:alnum:]]+=/\n&/g'

Leo: key value pair

2021-03-03T22:41:26Z

key value pair

← Older revision		Revision as of 22:41, 3 March 2021
Line 112:		Line 112:
	*<code>$!</code> if the last command failed (no more new lines?), branch to label a		*<code>$!</code> if the last command failed (no more new lines?), branch to label a
	*Otherwise, with the next line joined, we search and replace (like usual) by searcing for <code>\n</code> and replacing it with nothing.		*Otherwise, with the next line joined, we search and replace (like usual) by searcing for <code>\n</code> and replacing it with nothing.

			=== Place a key=value pair in its own line ===
			{{Highlight
			\| code = sed -r 's/[[:alnum:]]+=/\n&/g'
			\| lang = text
			}}

	==Further Reading==		==Further Reading==

Leo: /* Removing Newlines */

2020-10-23T23:40:27Z

Removing Newlines

← Older revision		Revision as of 23:40, 23 October 2020
Line 93:		Line 93:
	}}		}}

	== Removing Newlines ==		==Removing Newlines==
	If the output you have is wrapped with newlines indented in by a known amount, such as:		If the output you have is wrapped with newlines indented in by a known amount, such as:
	{{Highlight		{{Highlight
Line 106:		Line 106:
	Fri Jul 24 19:21:55: Dispatched 1 Task(s) on Host(s) <node008>, Allocated 1 Slot(s) on Host(s) <node008>, Effective RES_REQ <select[type == local] order[r15s:pg] >;		Fri Jul 24 19:21:55: Dispatched 1 Task(s) on Host(s) <node008>, Allocated 1 Slot(s) on Host(s) <node008>, Effective RES_REQ <select[type == local] order[r15s:pg] >;
	\| lang = terminal		\| lang = terminal
	}}		}}My understanding of how this works is:

	* <code>:a</code> creates a new		*<code>:a</code> creates a new label 'a'
			*<code>N</code> to join next line to the current line
			*<code>$!</code> if the last command failed (no more new lines?), branch to label a
			*Otherwise, with the next line joined, we search and replace (like usual) by searcing for <code>\n</code> and replacing it with nothing.

	==Further Reading==		==Further Reading==

Leo at 23:37, 23 October 2020

2020-10-23T23:37:23Z

← Older revision		Revision as of 23:37, 23 October 2020
Line 1:		Line 1:

			==Modifying a File Directly==
	== Modifying a File Directly ==

	Using the above tips, you can modify a file directly by using the {{code\|-i}} parameter, with the filename as the second argument:		Using the above tips, you can modify a file directly by using the {{code\|-i}} parameter, with the filename as the second argument:
Line 13:		Line 12:
	}}		}}

	== Simple search / replace ==		==Simple search / replace==
	Global search and replace:		Global search and replace:
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 24:		Line 23:
	}}		}}

	== Remove HTML tags ==		==Remove HTML tags==

	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 30:		Line 29:
	}}		}}

	== Remove File Extensions ==		==Remove File Extensions==

	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 41:		Line 40:
	}}		}}

	== Delete Matching Line ==		==Delete Matching Line==

	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 47:		Line 46:
	}}		}}

	== Inserting a Line Before/After a Match ==		==Inserting a Line Before/After a Match==

	Use: {{code\|0,/<regex>/s//<replacement text where \0 is the found line>/}}		Use: {{code\|0,/<regex>/s//<replacement text where \0 is the found line>/}}
Line 76:		Line 75:
	}}		}}

	== Grap Specific Section using Regex ==		==Grap Specific Section using Regex==
	To grab a specific section of a string, for instance '3' in 'Server time offset: 3':		To grab a specific section of a string, for instance '3' in 'Server time offset: 3':

Line 88:		Line 87:
	}}		}}

	== Join Every Other Line ==		==Join Every Other Line==
	To join every other line, useful for making key:value pairs, use:		To join every other line, useful for making key:value pairs, use:
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 94:		Line 93:
	}}		}}

	== Further Reading ==		== Removing Newlines ==
	* http://www.catonmat.net/blog/sed-one-liners-explained-part-one/		If the output you have is wrapped with newlines indented in by a known amount, such as:
			{{Highlight
			\| code = Fri Jul 24 19:21:55: Dispatched 1 Task(s) on Host(s) <node008>, Allocated 1 Slo
			t(s) on Host(s) <node008>, Effective RES_REQ <select[type
			== local] order[r15s:pg] >;
			\| lang = text
			}}
			You can use sed to replace a newline followed by a specific number of spaces to make this one whole line again:
			{{Highlight
			\| code = # cat above {{!}} sed ':a;N;$!ba;s/\n //g'
			Fri Jul 24 19:21:55: Dispatched 1 Task(s) on Host(s) <node008>, Allocated 1 Slot(s) on Host(s) <node008>, Effective RES_REQ <select[type == local] order[r15s:pg] >;
			\| lang = terminal
			}}

			* <code>:a</code> creates a new

			==Further Reading==

			*http://www.catonmat.net/blog/sed-one-liners-explained-part-one/

	{{Navbox Linux}}[[Category:Linux]]		{{Navbox Linux}}
			[[Category:Linux]]
	[[Category:TipsnTricks]]		[[Category:TipsnTricks]]
	[[Category:Code]]		[[Category:Code]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]

Leo: /* Modifying a File Directly */ multiple file glob

2019-09-12T05:34:09Z

Modifying a File Directly: multiple file glob

← Older revision		Revision as of 05:34, 12 September 2019
Line 5:		Line 5:
	Using the above tips, you can modify a file directly by using the {{code\|-i}} parameter, with the filename as the second argument:		Using the above tips, you can modify a file directly by using the {{code\|-i}} parameter, with the filename as the second argument:
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	$ sed -i "search" /path/to/file		$ sed -i "s/search/replace/g" /path/to/file
			}}

			You can also glob multiple files.
			{{highlight\|lang=terminal\|code=
			$ sed -i -e "s/search/replace/g" files*.txt
	}}		}}

Leo: formatting

2019-09-12T03:03:09Z

formatting

← Older revision		Revision as of 03:03, 12 September 2019
Line 4:		Line 4:

	Using the above tips, you can modify a file directly by using the {{code\|-i}} parameter, with the filename as the second argument:		Using the above tips, you can modify a file directly by using the {{code\|-i}} parameter, with the filename as the second argument:
	sed -i "search" /path/to/file		{{highlight\|lang=terminal\|code=
			$ sed -i "search" /path/to/file
			}}

	== Simple search / replace ==		== Simple search / replace ==
	Global search and replace:		Global search and replace:
	sed 's/<search>/<replace>/g'		{{highlight\|lang=terminal\|code=
			$ sed 's/<search>/<replace>/g'
			}}

	To replace only the first match:		To replace only the first match:
	sed '0,/<search>/s//<replace>/'		{{highlight\|lang=terminal\|code=
			$ sed '0,/<search>/s//<replace>/'
			}}

	== Remove HTML tags ==		== Remove HTML tags ==

	sed -e 's/<[^>]*>//g' <file>		{{highlight\|lang=terminal\|code=
			$ sed -e 's/<[^>]*>//g' <file>
			}}

	== Remove File Extensions ==		== Remove File Extensions ==

	sed 's/$.$\../\1/'		{{highlight\|lang=terminal\|code=
			$ sed 's/$.$\../\1/'
			}}

	eg:		eg:
	echo "house.txt" \| sed 's/$.$\../\1/' # returns 'house'		{{highlight\|lang=terminal\|code=
			$ echo "house.txt" {{!}} sed 's/$.$\../\1/' # returns 'house'
			}}

	== Delete Matching Line ==		== Delete Matching Line ==

	sed '/someline/d'		{{highlight\|lang=terminal\|code=
			$ sed '/someline/d'
			}}

	== Inserting a Line Before/After a Match ==		== Inserting a Line Before/After a Match ==

	Use: {{code\|0,/<regex>/s//<replacement text where \0 is the found line>/}}		Use: {{code\|0,/<regex>/s//<replacement text where \0 is the found line>/}}
	# Add a password line before the first title entry in the grub.conf file.		{{highlight\|lang=terminal\|code=
	sed -i "0,/^title/s//password --encrypted $Password\n\n\0/" $GrubConfig		## Add a password line before the first title entry in the grub.conf file.
			$ sed -i "0,/^title/s//password --encrypted $Password\n\n\0/" $GrubConfig
			}}

	To append after, just put \0 before your replacement text.		To append after, just put \0 before your replacement text.
Line 57:		Line 73:
	== Grap Specific Section using Regex ==		== Grap Specific Section using Regex ==
	To grab a specific section of a string, for instance '3' in 'Server time offset: 3':		To grab a specific section of a string, for instance '3' in 'Server time offset: 3':
	echo 'Server time offset: 3' \| sed -n -e 's/.: $[0-9]$/\1/p'
			{{highlight\|lang=terminal\|code=
			$ echo 'Server time offset: 3' {{!}} sed -n -e 's/.: $[0-9]$/\1/p'
			}}

	This can also be done with grep, though it isn't regex:		This can also be done with grep, though it isn't regex:
	echo 'Server time offset: 3' \| /bin/grep -ohE '[0-9]*'		{{highlight\|lang=terminal\|code=
			$ echo 'Server time offset: 3' {{!}} /bin/grep -ohE '[0-9]*'
			}}

	== Join Every Other Line ==		== Join Every Other Line ==
Line 67:		Line 88:
	'$!N;s/\n/ /'		'$!N;s/\n/ /'
	}}		}}


	== Further Reading ==		== Further Reading ==

Leo: Text replacement - "Category:Linux{{Navbox Linux}}" to "{{Navbox Linux}}Category:Linux"

2019-09-01T06:21:39Z

Text replacement - "Category:Linux{{Navbox Linux}}" to "{{Navbox Linux}}Category:Linux"

← Older revision		Revision as of 06:21, 1 September 2019
Line 72:		Line 72:
	* http://www.catonmat.net/blog/sed-one-liners-explained-part-one/		* http://www.catonmat.net/blog/sed-one-liners-explained-part-one/

	[[Category:Linux]]~~{{Navbox Linux}}~~		{{Navbox Linux}}[[Category:Linux]]
	[[Category:TipsnTricks]]		[[Category:TipsnTricks]]
	[[Category:Code]]		[[Category:Code]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]

Leo: Text replacement - "{{Navbox Linux}}{{Navbox Linux}}" to "{{Navbox Linux}}"

2019-09-01T05:10:09Z

Text replacement - "{{Navbox Linux}}{{Navbox Linux}}" to "{{Navbox Linux}}"

← Older revision		Revision as of 05:10, 1 September 2019
Line 72:		Line 72:
	* http://www.catonmat.net/blog/sed-one-liners-explained-part-one/		* http://www.catonmat.net/blog/sed-one-liners-explained-part-one/

	[[Category:Linux]]~~{{Navbox Linux}}~~{{Navbox Linux}}		[[Category:Linux]]{{Navbox Linux}}
	[[Category:TipsnTricks]]		[[Category:TipsnTricks]]
	[[Category:Code]]		[[Category:Code]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]

Leo: Text replacement - "Category:Linux" to "Category:Linux{{Navbox Linux}}"

2019-09-01T05:07:59Z

Text replacement - "Category:Linux" to "Category:Linux{{Navbox Linux}}"

← Older revision		Revision as of 05:07, 1 September 2019
Line 72:		Line 72:
	* http://www.catonmat.net/blog/sed-one-liners-explained-part-one/		* http://www.catonmat.net/blog/sed-one-liners-explained-part-one/

	[[Category:Linux]]{{Navbox Linux}}		[[Category:Linux]]{{Navbox Linux}}{{Navbox Linux}}
	[[Category:TipsnTricks]]		[[Category:TipsnTricks]]
	[[Category:Code]]		[[Category:Code]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]