Rsync: Difference between revisions

From Leo's Notes
This page was last edited on 10 March 2021, at 23:38.
No edit summary
No edit summary
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:


==Quick Usage Guide==
===Copying Files via SSH===
To copy files from {{code|/export/backup}} to a remote location (on server 'backup'):
{{highlight|lang=terminal|code=
$ rsync -avz /export/backup/ -e ssh backup:/export/backup/


== Quick Usage Guide ==
## Or with a non-standard SSH port
$ rsync -avz /export/backup/ -e "ssh -p 2222" backup:/export/backup/
}}


=== Copying Files via SSH ===
=== With sudo on remote server ===
To copy files from <code>/export/backup</code> to a remote location (on server 'backup'):
If you need to elevate your privileges with <code>sudo</code> when copying files on the remote system, specify an rsync path using sudo:
rsync -avz /export/backup/ -e ssh backup:/export/backup/
{{Highlight
| code = ## With sudo on the remote host
$ rsync -avz --rsync-path="sudo rsync"  -e "ssh -p 2222" user@remote:/path ./path
| lang = terminal
}}
You also need to add <code>rsync</code> in the <code>/etc/sudoers</code> file:
{{Highlight
| code = username ALL=NOPASSWD:/usr/bin/rsync
| lang = text
}}


=== Copying files with Progress Information ===
===Copying files with Progress Information===
rsync -avh --progress /src /dst
{{highlight|lang=terminal|code=
$ rsync -avh --progress /src/ /dst/
}}


== See Also ==
===Limiting Bandwidth===
* https://calomel.org/rsync_tips.html
The transfer rate can be limited with the {{code|1=--bwlimit=kbps}} option. Bytes per second is the default if no unit is given.
{{highlight|lang=terminal|code=
## Limit transfer speed to 1MB/s
$ rsync --bwlimit=1024 /source/ /destination/
$ rsync --bwlimit=1m /source/ /destination/
}}


===Parallel Copy===
You may wish to run multiple rsync in parallel to speed up data transfers. 
{{Highlight
| code = ## Transfers /source/directory tot /destination/directory with 5 parallel jobs
$ ls -1 /source/directory/ \
  {{!}} xargs -I {} -P 5 -n 1 rsync -avh --progress /source/directory/{}/ /destination/directory/{}/
| lang = terminal
}}


===Excluding a Directory===
If your source has snapshot directories (such as <code>.snapshots</code>) which aren't hidden by your filesystem by default, you can exclude it from being copied by using the <code>--exclude directory</code> option. Eg:
{{Highlight
| code = $ rsync --exclude .snapshots -avh --progress /source/ /destination/
| lang = terminal
}}


==Copying Directories==
Because I've been bitten by this issue way too many times in the past couple of days, to avoid copying a directory into another directory, '''always have a trailing slash from the source'''.
In more depth, suppose I have a {{code|src}} directory and I want its contents to be replicated in {{code|dst}}.
{{highlight|lang=terminal|code=
## This copies 'src' into 'dst/src'
$ rsync -avh src dst
## This too copies 'src' into 'dst/src'
$ rsync -avh src dst/
## Works as expected. Contents in src is now in dst
$ rsync -avh src/ dst
## Works as expected as well
$ rsync -avh src/ dst/
}}
==See Also==
*https://calomel.org/rsync_tips.html
{{Navbox Linux}}
[[Category:Linux]]
[[Category:Linux]]
[[Category:LinuxUtilities]]
[[Category:LinuxUtilities]]
[[Category:Todo]]
{{Navbox Linux}}

Latest revision as of 23:38, 10 March 2021

Quick Usage Guide

Copying Files via SSH

To copy files from /export/backup to a remote location (on server 'backup'):

$ rsync -avz /export/backup/ -e ssh backup:/export/backup/

## Or with a non-standard SSH port
$ rsync -avz /export/backup/ -e "ssh -p 2222" backup:/export/backup/

With sudo on remote server

If you need to elevate your privileges with sudo when copying files on the remote system, specify an rsync path using sudo:

## With sudo on the remote host
$ rsync -avz --rsync-path="sudo rsync"  -e "ssh -p 2222" user@remote:/path ./path

You also need to add rsync in the /etc/sudoers file:

username ALL=NOPASSWD:/usr/bin/rsync

Copying files with Progress Information

$ rsync -avh --progress /src/ /dst/

Limiting Bandwidth

The transfer rate can be limited with the --bwlimit=kbps option. Bytes per second is the default if no unit is given.

## Limit transfer speed to 1MB/s
$ rsync --bwlimit=1024 /source/ /destination/
$ rsync --bwlimit=1m /source/ /destination/

Parallel Copy

You may wish to run multiple rsync in parallel to speed up data transfers.

## Transfers /source/directory tot /destination/directory with 5 parallel jobs
$ ls -1 /source/directory/ \
   | xargs -I {} -P 5 -n 1 rsync -avh --progress /source/directory/{}/ /destination/directory/{}/

Excluding a Directory

If your source has snapshot directories (such as .snapshots) which aren't hidden by your filesystem by default, you can exclude it from being copied by using the --exclude directory option. Eg:

$ rsync --exclude .snapshots -avh --progress /source/ /destination/

Copying Directories

Because I've been bitten by this issue way too many times in the past couple of days, to avoid copying a directory into another directory, always have a trailing slash from the source.

In more depth, suppose I have a src directory and I want its contents to be replicated in dst.

## This copies 'src' into 'dst/src'
$ rsync -avh src dst

## This too copies 'src' into 'dst/src'
$ rsync -avh src dst/

## Works as expected. Contents in src is now in dst
$ rsync -avh src/ dst

## Works as expected as well
$ rsync -avh src/ dst/


See Also