rsync to remote server via ssh
March 1st, 2005If you ever need to repeatedly upload to a remote server, here is the command line of the day:
rsync -zrptL --delete-after -e "ssh" --include=core --include=tags --exclude=.DS_Store --cvs-exclude /local/dir user@host:/remote/dir/
rsync sends only the files that have changed. I just tried rsync today for the first time and I’m impressed. Its far faster than all of the ftp based synchronization tools I’ve used. Unfortunately, it only goes in one direction. Here are the meanings of the options:
z - use compression
r - recurse directories
p - preserve permissions
t - preserve times
L - copy (flatten) symbolic links
–delete-after - gets rid of any files in the remote directory that are not in the local directory. It prevents old files from getting orphaned on the server.
Put any ssh parameters you need inside the quotes.
I develop on Mac OS X, so –exclude=.DS_Store gets rid of those annoying little files.
–cvs-exclude gets rid of all the crud that CVS leaves lying around.
–include=core and –include=tags are my very hard won lesson of today. My program has a core and a tags directory. Tags is a special directory for cvs and removed by the –cvs-exclude option. Any file or directory named core is also removed by this option. –include puts em back. The other –cvs-exclude patterns are less likely to collide. I think –include has to come before the –cvs-exclude on the command line.
The trailing slash (or absence of one) on the source and destination directories matter.
All Posts
March 1st, 2005 at 4:30 pm
rsync works both ways; pushing and pulling data.
March 1st, 2005 at 7:03 pm
rsync goes up and down. To sync from remote to local just reverse the source and destination.
Try this link for a nice shell script and a modified version of rsync that can handle HFS+ resource forks. .
March 1st, 2005 at 7:45 pm
Take a look at Unison:
http://www.cis.upenn.edu/~bcpierce/unison/
It’s for synchronizing and also implements the rsync-algo for efficient bandwidth usage.
March 1st, 2005 at 8:44 pm
Look into Unison. It’s like rsync, but works in both directions.
http://www.cis.upenn.edu/~bcpierce/unison/
March 24th, 2005 at 4:10 pm
From the root of my project, I use:
rsync -avzu –exclude-from=rsync-exclude.txt * host:`pwd`
from my development server to the live server where the paths are the same.
I list a bunch of files in rsync-exclude.txt like images used only in production, files like CHANGELOG, etc. This file gets added to my CVS repository with each project.
I don’t use the –cvs-exclude because i tag my project before a release/upload and `cvs export -r TAGNAME project` so you won’t have CVS/ directories, etc.
The -n option is great and I encourage you use it before running it without. It will print you a list of all the files it wants to upload, allowing you to double check that you are about to upload anything by accident.
May 19th, 2005 at 12:10 am
–exclude-cvs considered harmful.
Well, it’s harmed me at least. Consider the case where you’re editing files on a ‘remote’ machine prior to committing them via a central machine ‘closer’ to the CVS repository(*).
Consider the following: you cvs update, rsync ‘away’ to another machine and do some editing there, cvs update again (whoops), then then rsync ‘back’ (disaster). You’ve now got ‘CVS/entries’ that say you’ve got the most recent files on the central system, but you’ve actually just rsync’d old versions of the files back. CVS will think you’re trying to ‘un-modify’ all the files back to where they were when you first CVS updated.
If you always keep the CVS meta files with the files they relate to, that sequence of events will simply result in CVS complaing you have state files.
Trust me. If you think you can simply just remember where you are and always update/sync in the right order, you’re kidding yourself. If you’re rsyncing checked-out CVS trees, its far too easy to mess up: blow the cost: sync your CVS meta data!
(*) pserver seems like a solution here, but consider the case where you need to test code on more than one platform *before* committing the code.