Copy data from a Synology Network Attached Storage (NAS) device to Petalibrary¶
CU Research Computing (CURC) PetaLibrary allocations can be accessed via the data transfer node (DTN) hosts using rsync over ssh. This document shows how to copy data from a Synology NAS device running DiskStation Manager (DSM) version 4.2 with hostname mylabstorage.colorado.edu to a PetaLibrary allocation named “peta_test” with RC account “RC_identikey.” A typical use case is where a university laboratory stores data on a local Synology server in the lab, and wants to back up the data to PetaLibrary.
Step 1: Connect to Synology via ssh¶
The following accounts are available for SSH remote login:
- DSM 6.0 and later: anyone in the administrators group
- DSM 5.2 and earlier/SRM: admin or root
Note: You may need to use the sudo command to gain administrative access.
$ ssh mylabstorage.colorado.edu -l admin firstname.lastname@example.org's password: BusyBox v1.16.1 (2013-04-16 20:13:10 CST) built-in shell (ash) Enter 'help' for a list of built-in commands. mylabstorage>
Step 2: Run ash to break out of restricted shell¶
mylabstorage> ash BusyBox v1.16.1 (2013-04-16 20:13:10 CST) built-in shell (ash) Enter 'help' for a list of built-in commands. /volume1/homes/admin $
Step 3: Use telnet to verify that we can talk to the CURC DTN ssh port¶
/volume1/homes/admin $ telnet dtn.rc.int.colorado.edu 22 _Note: if the above command does not yield an SSH banner (such as ‘SSH-2.0-OpenSSH_7.4’), ssh connectivity from the Synology to the DTN hosts is not working._
Note: These instructions assume you are on a CU Boulder campus network.
Step 4: Use ssh-keygen to generate an ssh key for connecting to PetaLibrary¶
(Note for below: Press return when prompted for a passphrase to select “empty”).
/volume1/homes/admin $ ssh-keygen -t rsa -b 4096 -C 'mylab synology' Generating public/private rsa key pair. Enter file in which to save the key (/var/services/homes/admin/.ssh/id_rsa): Created directory '/var/services/homes/admin/.ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /var/services/homes/admin/.ssh/id_rsa. Your public key has been saved in /var/services/homes/admin/.ssh/id_rsa.pub. The key fingerprint is: ...
Note that ssh-keygen will create two “keys”, one that is private and the other that is public. The private key will stay on the Synology and the public key will be copied to any host you want to access, in this case the CURC system. For those with interest in how SSH public key authentication works, see the ssh documentation.
Step 5: Add the public key from synology to CURC¶
$ cat .ssh/id_rsa.pub
This command will dump the contents of the public keys file to your screen. Now copy the new key to your clipboard.
$ ssh RC_identikey@login.rc.colorado.edu $ mkdir -p ~/.ssh
Now open the file ~/.ssh/authorized_keys in your favorite Linux editor (e.g., nano, vi or emacs) and paste output from your clipboard obtained from the cat command above. Save the file and exit.
Step 6: test connectivity from Synology to DTN¶
/volume1/homes/admin $ sftp RC_identikey@dtn.rc.int.colorado.edu
Note: if the above command does not yield an
sftp> prompt, ssh authentication from the Synology to the DTN hosts is not working.
Step 7: Transfer the data with rsync¶
$ rsync --protocol=29 -Havx "/volume1/mylabstorage/mylabfiles/." RC_identikey@dtn.rc.int.colorado.edu:/pl/active/peta_test/.
Note: If rsync does not produce any errors, log in to login.rc.colorado.edu and check that /pl/active/peta_test contains the files you wanted to copy.
Adjust the source and destination directories as appropriate. In the above example, any files in
/volume1/mylabstorage/mylabfiles on the Synology will be copied to
/pl/active/peta_test on the DTNs. You can add the
--delete argument to the rsync command, which will remove any files from /pl/active/peta_test that are not present in /volume1/mylabstorage/mylabfiles.