Rsync: How to transfer and remove the source files afterwards
Example how to move files with RSYNC
This articles is valid for all Unix flavours that have rsync available.
Rsync is one powerful tool for Linux administrators that allows one to synchronise two directories/paths based on source/destination model.
Among the many features it offers, there is one particularly useful when a migration (not backup) has to be performed and the source directory has to be wiped.
The rsync option that allows for source files and directories to be deleted is “--remove-source-files“.
But this is not enough for successful migration with rsync. In many situations, the owner/permissions and modification times of the source files and directories should be preserved on the destination directories. Here are a few options to accomplish this:
Code:
$ rsync 2>&1 | grep preserv
-H, --hard-links preserve hard links
-p, --perms preserve permissions
--executability preserve the file's executability
-o, --owner preserve owner (super-user only)
-g, --group preserve group
--devices preserve device files (super-user only)
--specials preserve special files
-t, --times preserve times
-O, --omit-dir-times omit directories when preserving times
Another set of rsync options offer recursively (to migrate directories instead of files), copy symlinks, preserve group and the “-a” option that combines all the above rsync options.
Code:
rsync 2>&1 | grep -E '^ \-(a|r|l|p|t|g|o|D)'
-a, --archive archive mode; same as -rlptgoD (no -H)
-r, --recursive recurse into directories
-l, --links copy symlinks as symlinks
-p, --perms preserve permissions
-o, --owner preserve owner (super-user only)
-g, --group preserve group
-D same as --devices --specials
-t, --times preserve times
$ rsync 2>&1 | grep -E remove-source
--remove-source-files sender removes synchronized files (non-dirs)
Delete source files/directories after rsync transfer is complete:
Before I provide the example, I should provide the environment details. Below, I will create a source directory having 4 sub directories and one file in each subdirectory. Then rsync will be used together with “—remove-source-files” and then I will show how the source and destination structure looks like.
Code:
acBook-Pro-van-Andrei:~ Andrei$ mkdir -p source/{1,2,3,4}
my-mac:~ Andrei$ touch source/{1,2,3,4}/files
my-mac:~ Andrei$ find source/ -type f
source//1/files
source//2/files
source//3/files
source//4/files
my-mac:~ Andrei$ mkdir destination
my-mac:~ Andrei$ rsync -rav --remove-source-files source/ destination/
building file list ... done
./
1/
1/files
2/
2/files
3/
3/files
4/
4/files
sent 359 bytes received 138 bytes 994.00 bytes/sec
total size is 0 speedup is 0.00
my-mac:~ Andrei$ find source/ -ls
1173801 0 drwxr-xr-x 6 Andrei staff 204 Jan 22 19:42 source/
1173802 0 drwxr-xr-x 2 Andrei staff 68 Jan 22 19:43 source//1
1173803 0 drwxr-xr-x 2 Andrei staff 68 Jan 22 19:43 source//2
1173804 0 drwxr-xr-x 2 Andrei staff 68 Jan 22 19:43 source//3
1173805 0 drwxr-xr-x 2 Andrei staff 68 Jan 22 19:43 source//4
my-mac:~ Andrei$ find destination/ -ls
1177476 0 drwxr-xr-x 6 Andrei staff 204 Jan 22 19:42 destination/
1183980 0 drwxr-xr-x 3 Andrei staff 102 Jan 22 19:42 destination//1
1183984 0 -rw-r--r-- 1 Andrei staff 0 Jan 22 19:42 destination//1/files
1183981 0 drwxr-xr-x 3 Andrei staff 102 Jan 22 19:42 destination//2
1183985 0 -rw-r--r-- 1 Andrei staff 0 Jan 22 19:42 destination//2/files
1183982 0 drwxr-xr-x 3 Andrei staff 102 Jan 22 19:42 destination//3
1183986 0 -rw-r--r-- 1 Andrei staff 0 Jan 22 19:42 destination//3/files
1183983 0 drwxr-xr-x 3 Andrei staff 102 Jan 22 19:42 destination//4
1183987 0 -rw-r--r-- 1 Andrei staff 0 Jan 22 19:42 destination//4/files
Let’s explain the above output: After rsync is used, only source files are deleted (not directories). This is described in rsync help. Next, the find commands show that the source directory has only the four subdirectories. The destination directory includes the 4 subdirectories as well as the files that were synchronised / migrated.
Few notes:
- Before running this live, these options should be tested in advance.
- Other rsync options like the below ones should be studies and not used unless really necessary:
Code:
-n, --dry-run show what would have been transferred
--delete delete extraneous files from destination dirs
--delete-before receiver deletes before transfer (default)
--delete-during receiver deletes during transfer, not before
--delete-after receiver deletes after transfer, not before
--delete-excluded also delete excluded files from destination dirs
If the rsync transfer is performed over the network, another option is useful to speed up the transfer:
Code:
-z, --compress compress file data during the transfer
And if the transfer speeds are required:
Code:
--progress show progress during transfer
News 
Site map 
SitemapIndex 
RSS Feed 