{"id":1026,"date":"2018-06-18T13:02:15","date_gmt":"2018-06-18T07:32:15","guid":{"rendered":"http:\/\/jaipurhosting.com\/blog\/?p=1026"},"modified":"2019-04-27T11:47:57","modified_gmt":"2019-04-27T06:17:57","slug":"how-to-transfer-files-using-rsync-command","status":"publish","type":"post","link":"https:\/\/www.jaipurhosting.com\/blog\/how-to-transfer-files-using-rsync-command\/","title":{"rendered":"How to transfer files using rsync command"},"content":{"rendered":"<p><strong>Description&nbsp;<\/strong><\/p>\n<p>Rsync (<strong>Remote Sync<\/strong>) is a most commonly used command for <strong>copying <\/strong>and <strong>synchronizing <em>files&nbsp;<\/em><\/strong>and <em><strong>directories<\/strong> <\/em>remotely as well as locally in Linux\/Unix systems.<\/p>\n<p>With the help of rsync command you can copy and synchronize your data <strong>remotely and locally<\/strong> across directories, across disks and networks, perform data backups and mirroring between two Linux machines.<\/p>\n<p><strong>Advantages and features of Rsync command&nbsp;<\/strong><\/p>\n<ul>\n<li>It efficiently <strong>copies <\/strong>and<strong> sync files<\/strong> to or from a remote system.<\/li>\n<li>Supports copying links, devices, owners, groups and permissions.<\/li>\n<li>It\u2019s faster than scp (<strong>Secure Copy<\/strong>) because rsync uses <strong>remote-update protocol<\/strong> which allows to transfer just the differences between two sets of files. First time, it copies the<strong> whole content<\/strong> of a file or a directory from source to destination but from next time, it copies only the <strong>changed blocks <\/strong>and <strong>bytes<\/strong> to the destination.<\/li>\n<li>Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.<\/li>\n<\/ul>\n<p><strong>Basic syntax of rsync command<\/strong><\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true\"># rsync options source destination<\/pre>\n<p><strong>Common options used with rsync commands<\/strong><\/p>\n<p><strong>-v<\/strong> : verbose<br \/>\n<strong>-r<\/strong> : copies data recursively (but don\u2019t preserve timestamps and permission while transferring data<br \/>\n<strong>-a<\/strong> : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user &amp; group ownerships and timestamps<br \/>\n<strong>-z<\/strong> : compress file data<br \/>\n<strong>-h<\/strong> : human-readable, output numbers in a human-readable format<\/p>\n<p>&nbsp;<\/p>\n<p><strong>Install rsync in Linux<\/strong><\/p>\n<p>We can install rsync package with the help of following command.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# yum install rsync<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>1.Copy\/Sync Files and Directory Locally<\/strong><\/p>\n<p><span style=\"color: #993300;\"><em><strong>Copy\/Sync a File on a Local Computer<\/strong><\/em><\/span><\/p>\n<p>This following command will sync a single file on a local machine from one location to another location.<\/p>\n<p>Example, a file name backup.tar needs to be copied or synced to <strong>\/tmp\/backups\/ folder.<\/strong><\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -zvh backup.tar \/tmp\/backups\/\ncreated directory \/tmp\/backups\nbackup.tar\nsent 14.71M bytes  received 31 bytes  3.27M bytes\/sec\ntotal size is 16.18M  speedup is 1.1<\/pre>\n<p><strong>Note<\/strong> : If the destination is not already exists, rsync will create a directory automatically for destination.<\/p>\n<p><span style=\"color: #993300;\"><em><strong>Copy\/Sync a Directory on Local Computer<\/strong><\/em><\/span><\/p>\n<p>The following command will transfer or sync all the files of from one directory to a different directory in the same machine.<\/p>\n<p>Example, \/root\/rpmpkgs contains some rpm package files and you want that directory to be copied inside<strong> \/tmp\/backups\/ folder<\/strong>.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avzh \/root\/rpmpkgs \/tmp\/backups\/\nsending incremental file list\nrpmpkgs\/\nrpmpkgs\/httpd-2.2.3-82.el5.centos.i386.rpm\nrpmpkgs\/mod_ssl-2.2.3-82.el5.centos.i386.rpm\nrpmpkgs\/nagios-3.5.0.tar.gz\nrpmpkgs\/nagios-plugins-1.4.16.tar.gz\nsent 4.99M bytes  received 92 bytes  3.33M bytes\/sec\ntotal size is 4.99M  speedup is 1.00<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>2.Copy\/Sync Files and Directory to or From a Server<\/strong><\/p>\n<p><strong><span style=\"color: #993300;\"><em>Copy a Directory from Local Server to a Remote Server<\/em><\/span><\/strong><\/p>\n<p>This command will sync a directory from a local machine to a remote machine.<\/p>\n<p>Example: There is a folder in your local computer \u201c<strong>rpmpkgs<\/strong>\u201d which contains some RPM packages and you want that local directory\u2019s content send to a remote server, you can use following command<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avz rpmpkgs\/ root@192.168.0.101:\/home\/\nroot@192.168.0.101's password:\nsending incremental file list\n.\/\nhttpd-2.2.3-82.el5.centos.i386.rpm\nmod_ssl-2.2.3-82.el5.centos.i386.rpm\nnagios-3.5.0.tar.gz\nnagios-plugins-1.4.16.tar.gz\nsent 4993369 bytes  received 91 bytes  399476.80 bytes\/sec\ntotal size is 4991313  speedup is 1.00<\/pre>\n<p>&nbsp;<\/p>\n<p><span style=\"color: #993300;\"><em><strong>Copy\/Sync a Remote Directory to a Local Machine<\/strong><\/em><\/span><\/p>\n<p>This command will help you sync a remote directory to a local directory.<\/p>\n<p>Example, a directory <strong>\/home\/folder\/rpmpkgs<\/strong> which is on a remote server is being copied in your local computer in <strong>\/tmp\/myrpms<\/strong>.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avzh root@192.168.0.100:\/home\/folder\/rpmpkgs \/tmp\/myrpms\nroot@192.168.0.100's password:\nreceiving incremental file list\ncreated directory \/tmp\/myrpms\nrpmpkgs\/\nrpmpkgs\/httpd-2.2.3-82.el5.centos.i386.rpm\nrpmpkgs\/mod_ssl-2.2.3-82.el5.centos.i386.rpm\nrpmpkgs\/nagios-3.5.0.tar.gz\nrpmpkgs\/nagios-plugins-1.4.16.tar.gz\nsent 91 bytes  received 4.99M bytes  322.16K bytes\/sec\ntotal size is 4.99M  speedup is 1.00<\/pre>\n<p><strong>3. Rsync Over SSH<\/strong><\/p>\n<p>With rsync, we can use SSH (<strong>Secure Shell<\/strong>) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet.<\/p>\n<p>Also when we use rsync we need to provide the user\/root password to accomplish that particular task, so using SSH option will send your logins in an encrypted manner so that your password will be safe<\/p>\n<p><span style=\"color: #993300;\"><em><strong>Copy a File from a Remote Server to a Local Server with SSH<\/strong><\/em><\/span><\/p>\n<p>To specify a protocol with rsync you need to give \u201c<strong>-e<\/strong>\u201d option with protocol name you want to use.<\/p>\n<p>Example, We will be using \u201c<strong>ssh<\/strong>\u201d with \u201c<strong>-e<\/strong>\u201d option and perform data transfer.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avzhe ssh root@192.168.0.100:\/root\/install.log \/tmp\/\nroot@192.168.0.100's password:\nreceiving incremental file list\ninstall.log\nsent 30 bytes  received 8.12K bytes  1.48K bytes\/sec\ntotal size is 30.74K  speedup is 3.77<\/pre>\n<p><span style=\"color: #993300;\"><em><strong>Copy a File from a Local Server to a Remote Server with SSH<\/strong><\/em><\/span><\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avzhe ssh backup.tar root@192.168.0.100:\/backups\/\nroot@192.168.0.100's password:\nsending incremental file list\nbackup.tar\nsent 14.71M bytes  received 31 bytes  1.28M bytes\/sec\ntotal size is 16.18M  speedup is 1.10<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>4. Show Progress While Transferring Data with rsync<\/strong><\/p>\n<p>To show the progress while transferring the data from one machine to a different machine, we can use \u2018<strong>\u2013progress<\/strong>\u2019 option for it.<\/p>\n<p>It displays the files and the time remaining to complete the transfer.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avzhe ssh --progress \/home\/rpmpkgs root@192.168.0.100:\/root\/rpmpkgs\nroot@192.168.0.100's password:\nsending incremental file list\ncreated directory \/root\/rpmpkgs\nrpmpkgs\/\nrpmpkgs\/httpd-2.2.3-82.el5.centos.i386.rpm\n1.02M 100%        2.72MB\/s        0:00:00 (xfer#1, to-check=3\/5)\nrpmpkgs\/mod_ssl-2.2.3-82.el5.centos.i386.rpm\n99.04K 100%  241.19kB\/s        0:00:00 (xfer#2, to-check=2\/5)\nrpmpkgs\/nagios-3.5.0.tar.gz\n1.79M 100%        1.56MB\/s        0:00:01 (xfer#3, to-check=1\/5)\nrpmpkgs\/nagios-plugins-1.4.16.tar.gz\n2.09M 100%        1.47MB\/s        0:00:01 (xfer#4, to-check=0\/5)\nsent 4.99M bytes  received 92 bytes  475.56K bytes\/sec\ntotal size is 4.99M  speedup is 1.00<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>5. Use of \u2013include and \u2013exclude Options<\/strong><\/p>\n<p>These two options allows us to <strong>include <\/strong>and <strong>exclude files<\/strong> by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don\u2019t want to be transferred.<\/p>\n<p>Example, <strong>rsync command<\/strong> will include those files and directory only which starts with \u2018R\u2019 and exclude all other files and directory.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avze ssh --include 'R*' --exclude '*' root@192.168.0.101:\/var\/lib\/rpm\/ \/root\/rpm\nroot@192.168.0.101's password:\nreceiving incremental file list\ncreated directory \/root\/rpm\n.\/\nRequirename\nRequireversion\nsent 67 bytes  received 167289 bytes  7438.04 bytes\/sec\ntotal size is 434176  speedup is 2.59<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>6. Use of \u2013delete Option<\/strong><\/p>\n<p>If a file or directory not exist at the source, but already exists at the destination, you might want to delete that existing file\/directory at the target while syncing .<\/p>\n<p>We can use \u2018<strong>\u2013delete<\/strong>\u2018 option to delete files that are not there in source directory.<\/p>\n<p>Source and target are in sync. Now creating new file <strong>test.txt<\/strong> at the target.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# touch test.txt\n[root@tecmint]# rsync -avz --delete root@192.168.0.100:\/var\/lib\/rpm\/ .\nPassword:\nreceiving file list ... done\ndeleting test.txt\n.\/\nsent 26 bytes  received 390 bytes  48.94 bytes\/sec\ntotal size is 45305958  speedup is 108908.55<\/pre>\n<p>Target has the new file called <strong>test.txt<\/strong>, when synchronize with the source with \u2018<strong>\u2013delete<\/strong>\u2018 option,&nbsp; it removed the file <strong>test.txt<\/strong>.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>7. Set the Max Size of Files to be Transferred<\/strong><\/p>\n<p>You can specify the Max file size to be transferred or sync. You can do it with \u201c<strong>\u2013max-size<\/strong>\u201d option.<\/p>\n<p>Example, Max file size is <strong>200k<\/strong>, so this command will transfer only those files which are equal or smaller than <strong>200k<\/strong>.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync -avzhe ssh --max-size='200k' \/var\/lib\/rpm\/ root@192.168.0.100:\/root\/tmprpm\nroot@192.168.0.100's password:\nsending incremental file list\ncreated directory \/root\/tmprpm\n.\/\nConflictname\nGroup\nInstalltid\nName\nProvideversion\nPubkeys\nRequireversion\nSha1header\nSigmd5\nTriggername\n__db.001\nsent 189.79K bytes  received 224 bytes  13.10K bytes\/sec\ntotal size is 38.08M  speedup is 200.43<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>8. Automatically Delete source Files after successful Transfer<\/strong><\/p>\n<p>Now, suppose you have a main web server and a data backup server, you created a daily backup and synced it with your backup server, now you don\u2019t want to keep that local copy of backup in your web server.<\/p>\n<p>So, will you wait for transfer to complete and then delete those local backup file manually? Of Course NO. This automatic deletion can be done using \u2018\u2013remove-source-files\u2018 option.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync --remove-source-files -zvh backup.tar \/tmp\/backups\/\nbackup.tar\nsent 14.71M bytes  received 31 bytes  4.20M bytes\/sec\ntotal size is 16.18M  speedup is 1.10\n[root@tecmint]# ll backup.tar\nls: backup.tar: No such file or directory<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>9. Do a Dry Run with rsync<\/strong><\/p>\n<p>If you are a newbie and using rsync and don\u2019t know what exactly your command going do.<\/p>\n<p>Rsync could really mess up the things in your destination folder and then doing an undo can be a tedious job.<\/p>\n<p>Use of this option will not make any changes only do a dry run of the command and shows the output of the command, if the output shows exactly same you want to do then you can remove \u2018\u2013<strong>dry-run<\/strong>\u2018 option from your command and run on the terminal.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync --dry-run --remove-source-files -zvh backup.tar \/tmp\/backups\/\nbackup.tar\nsent 35 bytes  received 15 bytes  100.00 bytes\/sec\ntotal size is 16.18M  speedup is 323584.00 (DRY RUN)<\/pre>\n<p>&nbsp;<\/p>\n<p><strong>10. Set Bandwidth Limit and Transfer File<\/strong><\/p>\n<p>You can set the bandwidth limit while transferring data from one machine to another machine with the the help of \u2018<strong>\u2013bwlimit<\/strong>\u2018 option.<\/p>\n<p>This options helps us to limit <strong>I\/O bandwidth<\/strong>.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]# rsync --bwlimit=100 -avzhe ssh  \/var\/lib\/rpm\/  root@192.168.0.100:\/root\/tmprpm\/\nroot@192.168.0.100's password:\nsending incremental file list\nsent 324 bytes  received 12 bytes  61.09 bytes\/sec\ntotal size is 38.08M  speedup is 113347.05<\/pre>\n<p>Also, by default rsync syncs changed <strong>blocks <\/strong>and <strong>bytes only<\/strong>, if you want explicitly want to sync whole file then you use \u2018<strong>-W<\/strong>\u2018 option with it.<\/p>\n<pre class=\"theme:dark-terminal lang:default decode:true \">[root@vps ~]#  rsync -zvhW backup.tar \/tmp\/backups\/backup.tar\nbackup.tar\nsent 14.71M bytes  received 31 bytes  3.27M bytes\/sec\ntotal size is 16.18M  speedup is 1.10<\/pre>\n<p>&nbsp;<\/p>\n<p>We hope you\u2019ve found this useful!<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Description&nbsp; Rsync (Remote Sync) is a most commonly used command for copying and synchronizing files&nbsp;and directories remotely as well as locally in Linux\/Unix systems. With the help of rsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1300,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-1026","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/posts\/1026","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/comments?post=1026"}],"version-history":[{"count":3,"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/posts\/1026\/revisions"}],"predecessor-version":[{"id":1548,"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/posts\/1026\/revisions\/1548"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/media\/1300"}],"wp:attachment":[{"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/media?parent=1026"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/categories?post=1026"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jaipurhosting.com\/blog\/wp-json\/wp\/v2\/tags?post=1026"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}