Home
Where to Start
News and Updates
Tutorials
Data Products
Data Access
Sky Coverage
Instruments
Data Flow
Algorithms
Glossary
Known Problems
Help and Feedback
Search

Rsync Help for SDSS DAS Users

Users of the SDSS Data Archive Server (DAS) who wish to obtain more than 10 files are required to use rsync, an open source utility that provides fast incremental file transfer. Rsync is available for download at the rsync website. Here we provide detailed instructions for installation and use on a variety of platforms.

The same instructions apply for mirroring all or parts of the data archive server through the rsync URL rsync://dr3@rsync.sdss.org/DR3.

Unix and linux

Rsync is often installed by default on Unix and Linux systems. It is included in Red Hat 7.2, for instance. If it is not installed on your system, or you would like to upgrade to the latest version, you can find links to the available downloads. We recommend downloading and compiling the source, rather than retrieving the binary.

The source can be downloaded directly from the rsync download area. You will need to gunzip and untar the file if your browser does not do so automatically. A directory called rsync-2.5.6 is created; just cd into that directory, and execute
./configure
This will create a Makefile and config.h appropriate for your system. Then type
make
to compile the code. Finally, to put rsync in your path, either add it by hand, or type
make install.

At this point, you are ready to use rsync. When you visit the DAS access page and request data, you can choose rsync as your download method; if you request more than 10 files this will be the default. When your data is ready, you will be asked to download a file called sdss-rsync.lis to a local directory where you want to save your data. This file contains a listing of all the data files to be pulled over by rsync. Then, change the working directory to where you want the downloaded directory tree to be placed and execute
rsync -vtrLPR --copy-unsafe-links --include-from=sdss-rsync.lis rsync://dr4@rsync.sdss.org/DR4/ ./
to retrieve your files. (If you use rsync for mirroring the archive, you can create your own include file, or point rsync directly to a specific directory if only one directory is to be downloaded.) You will be prompted for a password, which is simply "dr4". If you will be downloading often, you can set an environment variable of this password by adding
setenv RSYNC_PASSWORD dr4
to your .cshrc file, or
export RSYNC_PASSWORD=dr4
to your .bashrc file.

NOTE:If you are behind a firewall, you will need to
setenv RSYNC_PROXY host:port
(or the equivalent export in non-csh-like shells) to point to your institution's proxy server and port number.

Windows

Windows users cannot download and run rsync directly. You will first have to install cygwin, a linux emulator. It works with all non-beta, non "release candidate", ix86 versions of Windows since Windows 95, with the exception of Windows CE. Cygwin is available for download here, free of charge. Click on the "Install Now" link, and a file called setup.exe will be downloaded. Running this will allow you to install Cygwin.

Because rsync is not included with Cygwin, you will also need to download and install rsync. Use your web browser to download the rsync source code. Open a Cygwin session, cd to the location of the source, and follow the instructions in the Unix section above to install rsync.

Macintosh

Mac users with OSX can follow the instructions in the Unix section above to download and install rsync. Users with OS9.x will unfortunately not be able to use this utility.


Last modified: Fri Oct 1 13:40:32 CDT 2004