Jupyter Hub

Niagara has a node which has been designated a Jupyter Hub. This node can be used to run your Juptyer Notebook sessions.

Jupyter Hub node on Niagara

The Niagara Jupyter Hub server is a single node, with 93 GB of memory and 20 cores. You can access the node directly, from outside of SciNet, via jupyter.scinet.utoronto.ca.

• Point your browser to 'jupyter.scinet.utoronto.ca' and log in with your Compute Canada account.
• The browser should now show the files in your $HOME on Niagara. (If not, try reloading the page, it may have timed out).
• To see your files on $SCRATCH, you need to have a symbolic link to $SCRATCH in your $HOME folder. This can be done by typing, once, in a terminal:

   ln -sT $SCRATCH $HOME/scratch

• You can open or create Python 2, Python 3, and R notebooks.
• Many python packages are already pre-installed.

Tips to get started

• Jupyter can also browse your (Niagara) files and edit them.
• Use the 'new' button to create a new python notebook.
• Click on the 'Conda' tab to see your existing conda environments.
• Give your notebooks reasonable names.
• To execute a Python input line, press `Shift-Enter`.
• Save your work periodically (even though there is autosave).
• To work similarly to `ipython --pylab`, do:

In [1]: from pylab import *
        %matplotlib notebook

Using the Jupyter Hub responsibly

Jupyter notebooks are a useful environment for data exploration, pipeline development, and other hands-on work. Such notebooks are not, however, intended for heavy production data crunching. If you need to do heavy data crunching you should develop a script and run such work on the compute nodes.

Furthermore, the Jupyter Hub is a shared resource. Other users will be using the node at the same time you are using it. Please do not use more than a few cores at a time, and do not use an excessive amount of memory.

Advantages and Disadvantages of a Notebook Environment

Drawbacks:

• Notebook files (.ipynb) are not scripts.
• Notebooks do not (always) work well with version control.
• The environment is designed to run in browser.
• The back-end runs on shared resources.
• Graphics are inline, which is great for quick exploration but make tweaking a plot harder (IPython+X works better for this).
• You can jump around in the notebook, and execute different parts: it can be hard to keep track of what you did.

Advantages:

• You can jump around in the notebook, and execute different parts: Easier exploration, experimentation and debugging.
• Auto-save.
• You can rerun parts of your code (while, e.g., keeping large data in memory)
• You can add text portions, making your notebook more like an article.
• Which in turn can be useful for sharing, demos, teaching, ...
• You can still export as a script.
• Also has a terminal.