Mounting a remote filesystem from a CUmulus Virtual Machine

Learning Objectives:

Introduction

Remote filesystems on CURC can be mounted as filesystems on CUmulus VMs. This enables access to large datasets that may not otherwise be practical to place on the CUmulus VM, where storage is more limited. This tutorial demonstrates this capability. As a prerequisite, it is assumed you have already “spun-up” a CUmulus VM, as covered in Tutorial 1.

Tutorial

Completing the steps below requires that you have first logged into your VM


Part 1: Install sshfs

The sshfs software package provides an open-source solution for mounting your Petalibrary allocation to your CUmulus VM.

If you are on an Ubuntu or Debian VM, sshfs can be installed as follows:

sudo apt-get install sshfs

…or, if you are on a Centos or Redhat VM, sshfs can be installed as follows:

yum install fuse-sshfs

…or, if you are on a Windows VM, see our documentation here


Part 2: Mount PetaLibrary with sshfs

Create a folder on the VM where you would like to mount your PetaLibrary allocation. For simplicity, we will create a folder called mypetalibrary in our home (~) directory, but you could place the directory anywhere and call it anything you want:

mkdir ~/mypetalibrary

Now mount your PetaLibrary allocation to the directory. The generic syntax is:

sshfs <your-rc-username>@dtn.rc.int.colorado.edu:/pl/active/<your-pl-directory> <local-directory>

For example if your CURC username is ralphie and you want to mount your PetaLibrary allocation called rcops to ~/mypetalibrary:

sshfs ralphie@dtn.rc.int.colorado.edu:/pl/active/rcops ~mypetalibrary

You will be prompted for your password after execution. Type your password and accept the duo prompt. Your selected directory will now be mounted to your Petalibrary allocation.


Part 3: Test your mount

Can you see the files in your PetaLibrary allocation?

ubuntu@198.59.83.67:$ ls -l ~/mypetalibrary/
drwxr-sr-x 1  411465 1000000        2 Jun  8  2021  gtest
drwxr-sr-x 1  416810 1000000        8 Feb 20  2020  ior-testing
-rw-r--r-- 1 root    root     4194304 Oct  8  2020  testfile

…yes!

Can you copy a file from the PetaLibrary allocation?

ubuntu@198.59.83.67:$ cp ~/mypetalibrary/testfile .
ubuntu@198.59.83.67:$ ls -l .
total 4100
drwxrws--x 1 1000000 1000000      70 Mar  2 16:07 mypetalibrary
-rw-r--r-- 1 ubuntu  ubuntu  4194304 Mar 11 19:13 testfile

…yes! Can you create a new file and copy it to the PetaLibrary allocation?

ubuntu@198.59.83.67:$ touch newtestfile
ubuntu@198.59.83.67:$ cp newtestfile ~/mypetalibrary/
ubuntu@198.59.83.67:$ ls -l ~/mypetalibrary/newtestfile
-rw-r--r-- 1 122971 1000000 0 Mar 11 19:14 /home/ubuntu/mypetalibrary/newtestfile

…yes! Note that the user and group IDs (UID/GID) don’t map from CURC to CUmulus by default. so these will show up as numbers that correspond with the username under which you established the ssh mount (i.e., 122971 maps to ralphie in the example above).

Part 4: Remove your mount

Should you need to remove your mount, type the following, for example:

sshfs umount ~/mypetalibrary

Additional Information


This work has been funded in part by the National Science Foundation under grant OAC-1925766