Rsync - Revision history

Leo at 23:38, 10 March 2021

2021-03-10T23:38:12Z

← Older revision		Revision as of 23:38, 10 March 2021
Line 8:		Line 8:
	## Or with a non-standard SSH port		## Or with a non-standard SSH port
	$ rsync -avz /export/backup/ -e "ssh -p 2222" backup:/export/backup/		$ rsync -avz /export/backup/ -e "ssh -p 2222" backup:/export/backup/
			}}

			=== With sudo on remote server ===
			If you need to elevate your privileges with <code>sudo</code> when copying files on the remote system, specify an rsync path using sudo:
			{{Highlight
			\| code = ## With sudo on the remote host
			$ rsync -avz --rsync-path="sudo rsync" -e "ssh -p 2222" user@remote:/path ./path
			\| lang = terminal
			}}
			You also need to add <code>rsync</code> in the <code>/etc/sudoers</code> file:
			{{Highlight
			\| code = username ALL=NOPASSWD:/usr/bin/rsync
			\| lang = text
	}}		}}

Line 32:		Line 45:
	}}		}}

	=== Excluding a Directory ===		===Excluding a Directory===
	If your source has snapshot directories (such as <code>.snapshots</code>) which aren't hidden by your filesystem by default, you can exclude it from being copied by using the <code>--exclude directory</code> option. Eg:		If your source has snapshot directories (such as <code>.snapshots</code>) which aren't hidden by your filesystem by default, you can exclude it from being copied by using the <code>--exclude directory</code> option. Eg:
	{{Highlight		{{Highlight

Leo at 20:59, 18 January 2021

2021-01-18T20:59:28Z

← Older revision		Revision as of 20:59, 18 January 2021
Line 23:		Line 23:
	}}		}}

	=== Parallel Copy ===		===Parallel Copy===
	You may wish to run multiple rsync in parallel to speed up data transfers.		You may wish to run multiple rsync in parallel to speed up data transfers.
	{{Highlight		{{Highlight
	\| code = ## Transfers /source/directory tot /destination/directory with 5 parallel jobs		\| code = ## Transfers /source/directory tot /destination/directory with 5 parallel jobs
	$ ls -1 /source/directory/ \		$ ls -1 /source/directory/ \
	{{!}} xargs -I {} -P 5 -n 1 rsync -avh --progress /source/directory/{} /destination/directory/		{{!}} xargs -I {} -P 5 -n 1 rsync -avh --progress /source/directory/{}/ /destination/directory/{}/
			\| lang = terminal
			}}

			=== Excluding a Directory ===
			If your source has snapshot directories (such as <code>.snapshots</code>) which aren't hidden by your filesystem by default, you can exclude it from being copied by using the <code>--exclude directory</code> option. Eg:
			{{Highlight
			\| code = $ rsync --exclude .snapshots -avh --progress /source/ /destination/
	\| lang = terminal		\| lang = terminal
	}}		}}

Leo: /* Parallel Copy */ fixed bad pipe

2021-01-04T20:47:05Z

Parallel Copy: fixed bad pipe

← Older revision		Revision as of 20:47, 4 January 2021
Line 28:		Line 28:
	\| code = ## Transfers /source/directory tot /destination/directory with 5 parallel jobs		\| code = ## Transfers /source/directory tot /destination/directory with 5 parallel jobs
	$ ls -1 /source/directory/ \		$ ls -1 /source/directory/ \
	\| xargs -I {} -P 5 -n 1 rsync -avh --progress /source/directory/{} /destination/directory/		{{!}} xargs -I {} -P 5 -n 1 rsync -avh --progress /source/directory/{} /destination/directory/
	\| lang = terminal		\| lang = terminal
	}}		}}

Leo: /* Copying Directories */

2020-06-29T21:10:39Z

Copying Directories

← Older revision		Revision as of 21:10, 29 June 2020
Line 1:		Line 1:

	== Quick Usage Guide ==		==Quick Usage Guide==
	=== Copying Files via SSH ===		===Copying Files via SSH===
	To copy files from {{code\|/export/backup}} to a remote location (on server 'backup'):		To copy files from {{code\|/export/backup}} to a remote location (on server 'backup'):
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 10:		Line 10:
	}}		}}

	=== Copying files with Progress Information ===		===Copying files with Progress Information===
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	$ rsync -avh --progress /src/ /dst/		$ rsync -avh --progress /src/ /dst/
	}}		}}

	=== Limiting Bandwidth ===		===Limiting Bandwidth===
	The transfer rate can be limited with the {{code\|1=--bwlimit=kbps}} option. Bytes per second is the default if no unit is given.		The transfer rate can be limited with the {{code\|1=--bwlimit=kbps}} option. Bytes per second is the default if no unit is given.
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
Line 23:		Line 23:
	}}		}}

	== Copying Directories ==		=== Parallel Copy ===
			You may wish to run multiple rsync in parallel to speed up data transfers.
			{{Highlight
			\| code = ## Transfers /source/directory tot /destination/directory with 5 parallel jobs
			$ ls -1 /source/directory/ \
			\| xargs -I {} -P 5 -n 1 rsync -avh --progress /source/directory/{} /destination/directory/
			\| lang = terminal
			}}

			==Copying Directories==
	Because I've been bitten by this issue way too many times in the past couple of days, to avoid copying a directory into another directory, '''always have a trailing slash from the source'''.		Because I've been bitten by this issue way too many times in the past couple of days, to avoid copying a directory into another directory, '''always have a trailing slash from the source'''.

Line 44:		Line 53:


	== See Also ==		==See Also==
	* https://calomel.org/rsync_tips.html
			*https://calomel.org/rsync_tips.html

	{{Navbox Linux}}[[Category:Linux]]		{{Navbox Linux}}
			[[Category:Linux]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]

Leo: /* Limiting Bandwidth */

2020-04-17T02:27:11Z

Limiting Bandwidth

← Older revision		Revision as of 02:27, 17 April 2020
Line 16:		Line 16:

	=== Limiting Bandwidth ===		=== Limiting Bandwidth ===
	The transfer rate can be limited with the {{code\|--bwlimit=kbps}} option. Bytes per second is the default if no unit is given.		The transfer rate can be limited with the {{code\|1=--bwlimit=kbps}} option. Bytes per second is the default if no unit is given.
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	## Limit transfer speed to 1MB/s		## Limit transfer speed to 1MB/s

Leo at 02:26, 17 April 2020

2020-04-17T02:26:48Z

← Older revision		Revision as of 02:26, 17 April 2020
Line 1:		Line 1:


	== Quick Usage Guide ==		== Quick Usage Guide ==

	=== Copying Files via SSH ===		=== Copying Files via SSH ===
	To copy files from {{code\|/export/backup}} to a remote location (on server 'backup'):		To copy files from {{code\|/export/backup}} to a remote location (on server 'backup'):
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	# rsync -avz /export/backup/ -e ssh backup:/export/backup/		$ rsync -avz /export/backup/ -e ssh backup:/export/backup/

	## Or with a non-standard SSH port		## Or with a non-standard SSH port
	# rsync -avz /export/backup/ -e "ssh -p 2222" backup:/export/backup/		$ rsync -avz /export/backup/ -e "ssh -p 2222" backup:/export/backup/
	}}		}}

	=== Copying files with Progress Information ===		=== Copying files with Progress Information ===
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	# rsync -avh --progress /src/ /dst/		$ rsync -avh --progress /src/ /dst/
			}}

			=== Limiting Bandwidth ===
			The transfer rate can be limited with the {{code\|--bwlimit=kbps}} option. Bytes per second is the default if no unit is given.
			{{highlight\|lang=terminal\|code=
			## Limit transfer speed to 1MB/s
			$ rsync --bwlimit=1024 /source/ /destination/
			$ rsync --bwlimit=1m /source/ /destination/
	}}		}}

	== Copying Directories ==		== Copying Directories ==
			Because I've been bitten by this issue way too many times in the past couple of days, to avoid copying a directory into another directory, '''always have a trailing slash from the source'''.

	~~Because I've been bitten by this issue way too many times in the past couple of days, to avoid copying a directory into another directory, always have a trailing slash from the source.~~		In more depth, suppose I have a {{code\|src}} directory and I want its contents to be replicated in {{code\|dst}}.

	In more depth, suppose I have a 'src' directory and I want its contents to be replicated in 'dst'.

	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	## This copies 'src' into 'dst/src'		## This copies 'src' into 'dst/src'
	# rsync -avh src dst		$ rsync -avh src dst

	## This too copies 'src' into 'dst/src'		## This too copies 'src' into 'dst/src'
	# rsync -avh src dst/		$ rsync -avh src dst/

	## Works as expected. Contents in src is now in dst		## Works as expected. Contents in src is now in dst
	# rsync -avh src/ dst		$ rsync -avh src/ dst

	## Works as expected as well		## Works as expected as well
	# rsync -avh src/ dst/		$ rsync -avh src/ dst/

	}}		}}






	== See Also ==		== See Also ==
	* https://calomel.org/rsync_tips.html		* https://calomel.org/rsync_tips.html



	{{Navbox Linux}}[[Category:Linux]]		{{Navbox Linux}}[[Category:Linux]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]
	~~[[Category:Todo]]~~

Leo: /* Copying Files via SSH */

2019-11-04T17:38:26Z

Copying Files via SSH

← Older revision		Revision as of 17:38, 4 November 2019
Line 9:		Line 9:

	## Or with a non-standard SSH port		## Or with a non-standard SSH port
	# rsync -avz /export/backup/ -e "ssh -p 2222 backup:/export/backup/"		# rsync -avz /export/backup/ -e "ssh -p 2222" backup:/export/backup/
	}}		}}

Leo: Text replacement - "Category:Linux{{Navbox Linux}}" to "{{Navbox Linux}}Category:Linux"

2019-09-01T06:21:19Z

Text replacement - "Category:Linux{{Navbox Linux}}" to "{{Navbox Linux}}Category:Linux"

← Older revision		Revision as of 06:21, 1 September 2019
Line 48:		Line 48:


	[[Category:Linux]]~~{{Navbox Linux}}~~		{{Navbox Linux}}[[Category:Linux]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]
	[[Category:Todo]]		[[Category:Todo]]

Leo: Text replacement - "{{Navbox Linux}}{{Navbox Linux}}" to "{{Navbox Linux}}"

2019-09-01T05:11:04Z

Text replacement - "{{Navbox Linux}}{{Navbox Linux}}" to "{{Navbox Linux}}"

← Older revision		Revision as of 05:11, 1 September 2019
Line 48:		Line 48:


	[[Category:Linux]]~~{{Navbox Linux}}~~{{Navbox Linux}}		[[Category:Linux]]{{Navbox Linux}}
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]
	[[Category:Todo]]		[[Category:Todo]]

Leo: Text replacement - "Category:Linux" to "Category:Linux{{Navbox Linux}}"

2019-09-01T05:09:01Z

Text replacement - "Category:Linux" to "Category:Linux{{Navbox Linux}}"

← Older revision		Revision as of 05:09, 1 September 2019
Line 48:		Line 48:


	[[Category:Linux]]{{Navbox Linux}}		[[Category:Linux]]{{Navbox Linux}}{{Navbox Linux}}
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]
	[[Category:Todo]]		[[Category:Todo]]