SSH
SSH (secure shell) is the only way to log into the systems at SciNet. It opens a secure, encrypted connection between your computer and SciNet. If you have a Linux or Mac OSX machine, you already have SSH installed; if you have a Windows machine, you will have to install additional software before logging into SciNet.
SSH For Linux or Mac OS X Users
Simple Login
To login to SciNet's Niagara cluster, open a terminal window and type:
ssh USERNAME@niagara.scinet.utoronto.ca
replacing USERNAME with your SciNet username. Once done, you will be logged into the login nodes at the SciNet data centre, as if you have a terminal from those machines on your desktop.
***Note*** if you chose a custom SSH key name, i.e. something other than the default names: id_dsa, id_ecdsa, id_ed25519 and id_rsa, you will need to use the -i option of ssh to specify the path to your private key via:
ssh -i /path/to/key USERNAME@niagara.scinet.utoronto.ca
More details about custom SSH key names can be found here.
Note that if your username is the same on both the machine you're logging in from and the scinet machines, you can drop the USERNAME@, as SSH by default will try to use the username on the machine you are logging in from.
Copying Files
The SSH protocol can be used for more than logging in remotely; it can also be used to copy files between machines.
To copy small files from your home computer to a subdirectory of your /scratch directory at SciNet, you would type from a terminal on your computer
scp filetocopy.txt USERNAME@niagara.scinet.utoronto.ca:/scratch/USERNAME/some_subdirectory/
Note that soon the location of your scratch directory will change, and you will have to type:
scp filetocopy.txt USERNAME@niagara.scinet.utoronto.ca:/scratch/G/GROUPNAME/USERNAME/some_subdirectory/
Similarly, to copy files back into your current directory, you would type
scp USERNAME@niagara.scinet.utoronto.ca:/scratch/G/GROUPNAME/USERNAME/my_dirs/myfile.txt .
***Note*** if you chose a custom SSH key name, i.e. something other than the default names: id_dsa, id_ecdsa, id_ed25519 and id_rsa, you will need to use the -i option of scp and specify the path to your private key before the file paths via:
scp -i /path/to/key filetocopy.txt USERNAME@niagara.scinet.utoronto.ca:/scratch/USERNAME/some_subdirectory/
More details about custom SSH key names can be found here.
The Data Management wiki page has much more information on doing large transfers efficiently.
SSH for Windows Users
To use SSH on Windows, you will have to install SSH software. SciNet recommends, roughly in order of preference:
- Cygwin is an entire linux-like environment for Windows. Using something like Cygwin is highly recommended if you are going to be interacting a lot with linux systems, as it will give you a development environment very similar to that on the systems you'll be using. Download and run setup.exe, and install any packages you think you'll need. Once this is done, you will have icons for terminals, including one saying something like "X11". From either of these, you'll be able to type ssh user@niagara.scinet.utoronto.ca as above; if you think you will need to pop up windows from SciNet machines (e.g., for displaying data or using Profiling Tools), you'll need to use the X11 terminal and type ssh -Y user@niagara.scinet.utoronto.ca. Other ssh tools such as scp will work as above.
- MobaXterm is a tabbed ssh client with some Cygwin tools all wrapped up into one executable.
- OpenSSH For Windows installs only those parts of Cygwin necessary to run SSH. Again, once installed, opening up one of the new terminals allows you to use SSH as in the Linux/Mac OSX section above, but X11 forwarding for displaying windows may not work.
- PuTTY is one of the better stand-alone SSH programs for windows. It is a small download, and is enough to get you logged into the SciNet machines. For advanced use like X11 forwarding however, you are better off using Cygwin. A related program, PSCP, can be used to copy files using a graphical user interface.
WARNING: Make sure you download putty from the official website, because there are "trojanized" versions of putty around that will send your login information to a site in Russia (as reported here).
Copying Files
To transfer files to Niagara in Windows it is recommended to use the tool called WinSCP. Setting up a connection to Niagara using your SSH key can be done by following the steps in this link.
Copying Files with scp
If you want to use scp in Windows (e.g. MobaXterm terminal) and your SSH key was generated in MobaXterm/PuTTY, i.e. your private key has an .ppk extension, then you will need to convert your private key to an OpenSSH format. A good guide on how to do this can be found here.
X11 Forwarding
If during your login session you will only need to be typing and reading text, the techniques described above will suffice. However, if in a session you will need to be displaying graphics — such as plotting data on the scinet machines or using our performance profiling tools — you can use SSH's very powerful ability to forward several different types of data over one connection. To enable "X11 forwarding" over this SSH connection, add the option -Y to your command,
ssh -Y USERNAME@niagara.scinet.utoronto.ca
- Both, Windows and Mac OS users, will need to install an additional program to have X-forwarding working, usually referred to as "Xserver" which will interprete the data (graphics) forwarded and displayed on the local computer.
- Mac OS users need to install XQUARTZ
- Windows users could opt for installing MobaXterm which is a ssh-client which already includes an Xserver.
SSH Keys
SSH has an alternative to passwords to authenticate your login; you can generate a key file on a trusted machine and tell a remote machine to trust logins from a machine that presents that key. This can be both convenient and secure, and may be necessary for some tasks (such as connecting directly to compute nodes to use some visualization packages). Here we describe how to setup keys for logging into SciNet.
In addition to using passwords to authenticate users, one can use cryptographically secure keys to guarantee that a login request is coming from a trusted account on a remote machine, and automatically allow such requests. Done properly, this is as secure as requiring a password, but can be more convenient, and is necessary for some operations.
How SSH keys work
SSH relies on public key cryptography for its encryption. These cryptosystems have a private key, which must be kept secret, and a public key, which may be disseminated freely. In these systems, anyone may use the public key to encode a message; but only the owner of the private key can decode the message. This can also be used to verify identities; if someone is claiming to be Alice, the owner of some private key, Bob can send Alice a message encoded with Alice's well-known public key. If the person claiming to be Alice can then tell Bob what the message really was, then that person at the very least has access to Alice's private key.
To use keys for authentication, you need to:
- Generate a key pair (private and public)
- Copy the public key to a remote site, and add it to the list of authorized keys
- Ensure permissions are set properly
- Test to make sure it works
Generating an SSH key pair
The first stage is to create an SSH key pair. On Linux & MacOS (and Windows, with MobaXterm terminal) this is done using the ssh-keygen command:
ssh-keygen -t ed25519
If that doesn't work, try:
ssh-keygen -t rsa -b 4096
This will prompt you for two pieces of information: where to save the key, and a passphrase for the key. The passphrase is like a password, but rather than letting you in to some particular account, it allows you to use the key you've generated to log into other systems.
The default location to save the private key is in ${HOME}/.ssh/id_type (where type is ed25519 for an Ed25519 key or rsa for an RSA key); unless you have some specific reason for placing it elsewhere, use this option. The public key will be id_type.pub in the same directory.
While the default names for the keys are sufficient for most use cases, it is worth noticing that you are free to name them as you wish. For example, you might want to include a string referring to your name and where you plan on using this key like js_cc_ed25519 (and js_cc_ed25519.pub), if your name is John Smith and you created a key of ed25519 type on your laptop to be used on Alliance (formerly Compute Canada) HPC systems. This is useful to distinguish amongst several keys you might need to have on the same computer (for example different keys on the same computer to access different clusters).
Your passphrase can be any string, and of any length. It is best not to make it the same as any of your passwords. A reasonably strong passphrase usually constitutes of one to three sentences with a few words each, written with the proper punctuation in place. Make sure it is unique and memorable to you. Do not use any popular catch phrases, jingles, or song choruses. They could be easily found on the web and catalogued on a database for a brute-force attack.
A sample session of generating a key would go like this:
$ ssh-keygen -t ed25519 Generating public/private ed25519 key pair. Enter file in which to save the key (/home/USERNAME/.ssh/id_ed25519): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/USERNAME/.ssh/id_ed25519. Your public key has been saved in /home/USERNAME/.ssh/id_ed25519.pub. The key fingerprint is: SHA256:EajOndriRmLpl1qKg03FDhnc0EzRaApdBTygEbpQZrA USERNAME@HOSTNAME The key's randomart image is: +--[ED25519 256]--+ |+=*X=*... | |oB+ O o . | |E. * o . | |..+ + . | |. B . .S | | = = o | |.= o.+ | |o.oo* . | |..o=.. | +----[SHA256]-----+
Don't Use Passphraseless Keys!
If you do not specify a passphrase, you will have a completely "exposed" private key. This is a terrible idea. If you then use this key for anything it means that anyone who sits down at your desk, or anyone who borrows or steals your laptop, can login to anywhere you use that key (good guesses could come from just looking at your history) without needing any password, and could do anything they wanted with your account or data. Don't use passphraseless keys.
Don't Copy Your Private Key to Other Systems!
A private key should never leave the computer that was used to generate it. If you have a personal computer at home, a laptop and a desktop at work, for example, make sure to repeat the process for generating a public-private key pair on each of these systems. Insert a comment into the key so that you know where the key pair was created and how it was named. It is useful when distinguishing keys coming from different devices when inspecting the public keys. For example, issue the following commands:
$ ssh-keygen -t ed25519 -C "js@mylaptop js_cc" -f $HOME/.ssh/js_cc_ed25519 $ ssh-keygen -t ed25519 -C "js@mydesktop js_cc" -f $HOME/.ssh/js_cc_ed25519 $ ssh-keygen -t ed25519 -C "js@workdesktop js_cc" -f $HOME/.ssh/js_cc_ed25519
for keys created on your personal laptop, desktop and work desktop, respectively (Replace the suggested "js" abbreviations above, or any other reference to John Smith name, to yours). See Multiple SSH keys session for other examples.
Private keys are very powerful. In the wrong hands, they can be used to impersonate you on every system you use those keys to access to. Copying them from the location where they were generated to other places also increases the number of systems that that key can be used to compromise. In addition, leaving the private key where it was created protects you against man-in-the-middle attacks, which impersonate the systems you want to copy your keys to and steal your private key in the process.
Uploading the Public Key to CCDB
Use your Alliance (formerly Compute Canada) credentials to visit the following site:
https://ccdb.computecanada.ca/ssh_authorized_keys
and follow the instructions there to upload your public key. By using the 'cat' command as follows:
cat ~/.ssh/id_ed25519.pub
you can inspect, select and copy the plain text output of your public key to your screen into the CCDB web page above.
Keys uploaded to the CCDB are available by all clusters across the Digital Reseach Alliance of Canada (formerly Compute Canada).
~/.ssh Directory Permissions
Note that SSH is very fussy about file permissions; your ~/.ssh directory must only be accessible by you, and your various key files must not be writable (or in some cases, readable) by anyone else. Sometimes users accidentally reset file permissions while editing these files, and problems happen. If you look at the ~/.ssh directory itself, it should not be accessible by anyone else:
$ ls -ld ~/.ssh drwx------ 2 USERNAME GROUPNAME 7 Aug 9 15:43 /home/.../.ssh
To fix your permissions, use the following command:
chmod -R go= ~/.ssh/
Testing Your Key
Now you should be able to login to the remote system (say, SciNet):
$ ssh USERNAME@niagara.scinet.utoronto.ca Enter passphrase for key '/home/USERNAME/.ssh/id_ed25519': Last login: Tue Aug 17 11:24:48 2010 from HOSTNAME [...] nia-login07-$
If this is indeed the absolute first time you are trying to access Niagara, for example, please make sure you are actually accessing Niagara by double checking Niagara's login node ssh key fingerprint as instructed here. This check is important to avoid being victim of man-in-the-middle attacks. As a reference you can check host key fingerprints for other Alliance (formerly Compute Canada) systems at this link.
If you get the message below you may need to logout of your gnome session and log back in since ssh-agent needs to be restarted with the new passphrase ssh key.
$ ssh USERNAME@niagara.scinet.utoronto.ca Agent admitted failure to sign using the key.
(Optional) Using ssh-agent to Remember Your Key
But now you've just replaced having to type a password for login with having to type a passphrase for your key; what have you gained?
It turns out that there's an automated way to manage ssh "identities", using the ssh-agent command, which should automatically be running on newer Linux or macOS machines. You can add keys to this agent for the duration of your login using the ssh-add command:
$ ssh-add Enter passphrase for /home/USERNAME/.ssh/id_ed25519: Identity added: /home/USERNAME/.ssh/id_ed25519 (/home/USERNAME/.ssh/id_ed25519)
and then logins will not require the passphrase, as ssh-agent will provide access to the key.
When you log out of your home computer, the ssh agent will close, and next time you log in, you will have to ssh-add your key. You can also set a timeout of (say) an hour by using ssh-add -t 3600. This minimizes the number of times you have to type your passphrase, while still maintaining some degree of key security.
You can list the fingerprints of all identities currently represented by the agent with '-l' option: 'ssh-add -l'. To delete all identities from the agent just type 'ssh-add -D'.
Custom SSH Keys names
If you use a custom name for your SSH key pair, you will need to create or modify the config file under ~/.ssh/config to something similar:
Host niagara HostName niagara.scinet.utoronto.ca User YOUR_LOGIN IdentityFile ~/.ssh/ssh_privatekey_name
Then authenticate by typing:
$ssh niagara
You can also use the -i option of ssh to specify the path to your private key via:
$ssh -i /path/to/key USERNAME@niagara.scinet.utoronto.ca
Multiple SSH keys
It's recommended to have different ssh keys for each service, specific role, or domain. For example, to have separate keys for niagara and graham, first generate two new keys:
$ ssh-keygen -t ed25519 -f ~/.ssh/id_niagara -C "Key for Niagara" ... $ ssh-keygen -t ed25519 -f ~/.ssh/id_graham -C "Key for Graham" ...
Make sure to use different file names for each key. Next, modify your ~/.ssh/config file, adding IdentityFile directives:
Host niagara User myusername HostName niagara.scinet.utoronto.ca IdentityFile ~/.ssh/id_niagara Host graham User myusername HostName graham.computecanada.ca IdentityFile ~/.ssh/id_graham
Now when you login with the shortcuts
$ ssh niagara
or
$ ssh graham
different keys will be used.
Examples
Copying a file on another cluster to Niagara
For convenience I will assume you have already pasted your public key on CCDB so you can access all clusters of the Digital Reseach Alliance of Canada (formerly Compute Canada) with that key. Your modified ~/.ssh/config file (on your linux laptop for example) could use then the same IdentityFile directive for all clusters. In this example, let's say Graham and Niagara on the Alliance (formerly Compute Canada) use the same key, as well as a remote cluster not part of the Alliance (formerly Compute Canada):
Host niagara User myusername HostName niagara.scinet.utoronto.ca IdentityFile ~/.ssh/id_ed25519 Host graham User myusername HostName graham.computecanada.ca IdentityFile ~/.ssh/id_ed25519 Host remote_cluster User myusername HostName remote_cluster.other_domain.ca IdentityFile ~/.ssh/id_ed25519
Suppose you want to access Niagara and copy a file on Graham there, or a file on remote_cluster. How would you do it? Based on the assumptions above you could proceed as follows:
1) On your laptop, the first step to follow is to load your private key to the ssh-agent daemon:
$ ssh-add /home/USERNAME/.ssh/id_ed25519 Enter passphrase for /home/USERNAME/.ssh/id_ed25519: Identity added: /home/USERNAME/.ssh/id_ed25519 (/home/USERNAME/.ssh/id_ed25519)
It is very convenient since you do it once per work session only and it is strongly encouraged.
2) Access Niagara and enable forwarding of the authentication agent connection (ssh-agent):
$ ssh -A niagara
3) Use secure copy command to copy the file on Graham to Niagara:
$ scp -p graham.computecanada.ca:/path/to/file.txt .
'-p' option is commonly used to preserve the date stamp and other file attributes as it is copied. Not mandatory though. Use the '-R' if you want to copy a directory recursively. Be aware though that it might be better to create a tarball first if there are lots of small files in the directory. Note that if you want to use an alias for graham in the scp command as follows:
$ scp -p graham:/path/to/file.txt .
you would have to copy your modified ~/.ssh/config file above from your laptop to Niagara first.
4) Use secure copy command to copy the file on remote_cluster to Niagara:
$ scp -p remote_cluster.other_domain.ca:/path/to/file.txt .
This assumes you have copied your public key from your laptop to the remote_cluster first:
$ ssh-copy-id -i ~/.ssh/id_ed25519.pub remote_cluster
or in the absence of ssh-copy-id command:
$ cat ~/.ssh/id_ed25519.pub | ssh remote_cluster "cat >> ~/.ssh/authorized_keys"
and also assumes remote_cluster is not part of the Alliance (formerly Compute Canada): therefore it doesn't have a similar mechanism to manage user's ssh keys centrally as CCDB does.
5) The '-A' option should be used deliberately for this copying task for the shortest time needed. The rationale is that if Niagara is compromised an attacker would now be able to use your connection to access other systems using your forwarded ssh-agent. Better to exit and return without any option added to ssh:
$ exit $ ssh niagara
Best Practice Summary
* Create one key pair for each computer you work on and give it a meaningful name. In addition, a comment can also help identify the device where they were created. * Protect each of your private keys with a strong passphrase. We recommend fifteen characters or more. * Do not share your private keys. * Never copy your private keys to other systems. * Create one key pair for each different service, role or domain, and name them accordingly. * Do not create key pairs in shared systems like HPC clusters.
SSH Tunnels
A more-obscure technique for setting up SSH communication is the construction of an SSH tunnel. This can be useful if, for example, your code needs to access an external software license server from a Niagara compute node. You can read about setting up SSH tunnels on Niagara here.
Multifactor authentication
Multifactor authentication via Duo or Yubikey is now required for all users. Please visit the CCDB to enrol, and more information can be found here.