Globus

Copying Files to Niagara From the Linux Command-line

Step 1: Install globus CLI

Requires python 2.7. On the machine you're transferring from:

$ virtualenv venv-globus
$ source ./venv-globus/bin/activate
$ pip install globus-cli

Step 2: Login to globus

$ globus login

You should see:

Please log into Globus here:
---------------------------
https://auth.globus.org/v2/oauth2/...
---------------------------

Enter the resulting Authorization Code here:

Visit the URL in a web browser, choose "Compute Canada" as your organization, and enter your Niagara username/password.

Step 3: Create a personal endpoint

$ globus endpoint create --personal my-endpoint-name

Replace my-endpoint-name with a name of your choice.

You should see:

Message:     Endpoint created successfully
Endpoint ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Setup Key:   yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy

Save this info as we'll need it later.

Step 4: Get Globus Connect Personal

$ wget https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz
$ tar -xzf globusconnectpersonal-latest.tgz
$ cd globusconnectpersonal-x.y.z

Step 5: Setup your endpoint

$ ./globusconnectpersonal -setup yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy

replacing "yyyy..." with the Setup Key from 'globus endpoint create'. You should see something like:

Configuration directory: /home/username/.globusonline/lta
Contacting relay.globusonline.org:2223
Done!

Step 6: Configure your endpoint (optional)

By default globus only allows transfers to/from your home directory. Edit ~/.globusonline/lta/config-paths and add a line for any other directories you need, e.g.:

/path/to/data/,0,1

Step 7: Start Globus Connect

$ ./globusconnectpersonal -start &

Step 8: Set some convenience variables

$ my_endpoint="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
$ niagara_endpoint="77506016-4a51-11e8-8f88-0a6d4e044368"

replacing "xxxx..." with the Endpoint ID from 'globus endpoint create'.

Step 9: Activate the Niagara endpoint

$ globus endpoint activate --myproxy --myproxy-lifetime=1000 $niagara_endpoint

Step 10: Start a transfer

$ globus transfer --recursive $my_endpoint:/path/to/data $niagara_endpoint:/scratch/g/group/username/data