Ansys

The Ansys engineering simulation tools are installed in both the Niagara and CC software stacks.

Getting a license

Licenses are provided by CMC Microsystems. Canadian students and faculty can register at this page.

Once you have an account, you must contact CMC and tell them you want to use the Ansys tools on Niagara, and give them your SciNet username.

After confirming with CMC and your institution that your username is authorized to claim a license, you will need to establish a ssh tunnel on your submission script in order to access the license server from niagara compute nodes (which don't have direct access to the internet). Please obtain the proper server name and ports from CMC or directly from your institution, so you may fill the ssh tunnel settings on the script template further below.

You may also use a license server from the other GP clusters (beluga/narval, cedar or graham), since niagara doesn't have its own server. More information can be found here:

https://docs.computecanada.ca/wiki/ANSYS#Configuring_your_own_license_file

Running on Niagara

Commercial software modules are not available by default, and require a 'module use' command:

module use /scinet/niagara/software/commercial/modules
module load ansys/2023r1

Programs available:

• fluent
• ansysedt
• mapdl
• ...

You can use the Ansys graphical tools to set up your problem, but you cannot use the graphical tools to submit your job. The job must be submitted to the scheduler for running.

Setting up your .mw directory

Ansys will attempt to write to your $HOME/.mw directory. This will work when you are testing your workflow on the login nodes, because they can write to $HOME. However, recall that the compute nodes cannot write to the /home filesystem. If you attempt to run Ansys from a compute node using the default configuration, it will fail because Ansys cannot write to $HOME/.mw.

The solution is to create an alternative directory called $SCRATCH/.mw, and create a soft link from $HOME/.mw to $SCRATCH/.mw:

mkdir $SCRATCH/.mw
ln -s $SCRATCH/.mw $HOME/.mw

This will fool Ansys into thinking it is writing to $HOME/.mw, when in fact it is writing to $SCRATCH/.mw. This command only needs to be run once.

Running ansys

Example submission script for a job running on 1 node, with max walltime of 11 hours:

#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=40
#SBATCH --time=11:00:00
#SBATCH --job-name test

module use /scinet/niagara/software/commercial/modules
module load ansys/2023r1

# DIRECTORY TO RUN - $SLURM_SUBMIT_DIR is directory job was submitted from
cd $SLURM_SUBMIT_DIR

machines=$(srun bash -c 'hostname -s' | sort | uniq | awk '{print $1 ":" 40}' | paste -s -d ':')
ansys231 -b -j JOBNAME -dis -machines "$machines" -i ansys.in

Custom license server

To use your own license server for Ansys, you'll need to open up ssh tunnels to the required ports, and point Ansys to the tunnels. For example:

ssh nia-gw -fNL 1055:license.server:1055
ssh nia-gw -fNL 1056:license.server:1056
ssh nia-gw -fNL 2325:license.server:2325

export ANSYSLI_SERVERS=2325@localhost
export ANSYSLMD_LICENSE_FILE=1055@localhost
export LM_LICENSE_FILE=1055@localhost

Make sure your firewall allows connections from the following IP addresses and confirm with your license provider which ports should be used with the ssh tunnel:

142.150.188.71
142.150.188.72
142.150.188.73
142.150.188.74
142.150.188.75
142.150.188.76
142.150.188.77
142.150.188.78
142.1.174.227
142.1.174.228