Scinet-conda

The scinet-conda wrapper is a package created to address Anaconda's feature of creating zillions of files upon installation, which is not good for HPC file systems. It creates a virtual environment-like interface to Anaconda installed in a Singularity container, thus eliminating the problem of have numerous small files.

How to use scinet-conda

Activating scinet-conda

By default scinet-conda is not available. It can be put into your environment using the command

 $ source /scinet/conda/etc/profile.d/scinet-conda.sh
 

This command must be invoked at the beginning of each session to make the necessary commands available.

Managing conda environments

You can now create conda environments using a similar syntax to regular Anaconda:

 $ scinet-conda create -n myvenv python==3.7

Note that you must specify the Python version that you want. This will create an Singularity, container which contains Anaconda, in your $HOME directory.

To list the existing scinet-conda environment use the command

 $ scinet-conda env list
 myvenv
 $

To delete an existing scinet-conda environment you can use the command

 $ scinet-conda env remove -n myvenv

Activating your scinet-conda environment

To activate a scinet-conda environment, use the command:

 $ scinet-conda activate myvenv
 (myenv)$

This environment can now be used in the same way as a regular virtual environment.

Installing packages

Once the environment has been activated conda packages can be installed in your scinet-conda environment in the same manner as a regular conda environment.

 (my.venv)$ scinet-conda install six

To install a package from a specific channel, specify the channel in the usual way.

 (my.venv)$ scinet-conda install --channel dglteam dgl

If you need to install a package that is not available through conda channels you can always use pip

 (my.venv)$ scinet-conda pip install nose

Accessing the Singularity container

For some packages you need access to the internal environment of the Singularity container itself. This is access using the 'shell' command

 (my.venv)$ scinet-conda shell