The STAR-CCM+ engineering simulation tool is installed on the CC software stack.

Getting a license

Licenses are provided by Siemens.

You will need to create a $HOME/.licenses/starccm.lic file with following contents:

SERVER 127.0.0.1 ANY 1999
USE_SERVER

and establish the ssh tunnel to the server at Siemens as shown in the submission script template further below. Also be sure to define the license env vars prior to loading the modules.

Running using the CC installation

STAR-CCM+ 12.04.011

STAR-CCM+ is only available on the CC software stack. 
Note that the default CC StdEnv changed since Apr/01/2021. 
On Niagara you must run the following module commands:

To use older versions of some software you need to do this now:
module load CCEnv
module load StdEnv/2018.3
module load starccm/14.04.013-R8

Please change your submission script accordingly.

If you adopt version 15.04.010-R8 (the latest), then you may use:

module load CCEnv
module load StdEnv
module load starccm/15.04.010-R8

Setting up your .star-12.04.011 directory

STAR-CCM+ version 12.04.011 will attempt to write to your $HOME/.star-12.04.011 directory. This will work when you are testing your workflow on the login nodes, because they can write to $HOME. However, recall that the compute nodes cannot write to the /home filesystem. If you attempt to run STAR-CCM+ from a compute node using the default configuration, it will fail because STAR-CCM+ cannot write to $HOME/.star-12.04.011.

The solution is to create an alternative directory called $SCRATCH/.star-12.04.011, and create a soft link from $SCRATCH/.star-12.04.011 to $HOME/.star-12.04.011:

mkdir $SCRATCH/.star-12.04.011
ln -s $SCRATCH/.star-12.04.011 $HOME/.star-12.04.011

This will fool Ansys into thinking it is writing to $HOME/.star-12.04.011, when in fact it is writing to $SCRATCH/.star-12.04.011. This command only needs to be run once, though it needs to be run for each version of STAR-CCM+ you run.

Running STAR-CCM+

Example submission script for a job running on 1 node, with max walltime of 4 hours:

#!/bin/bash
#SBATCH --time=0-00:30        # Time limit: d-hh:mm
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=40
#SBATCH --mail-type=BEGIN
#SBATCH --mail-type=END

cd $SLURM_SUBMIT_DIR

ssh nia-gw -L 1999:flex.cd-adapco.com:1999 -L 2099:flex.cd-adapco.com:2099 -N -f
export LM_PROJECT='YOUR CD-ADAPCO PROJECT ID GOES HERE'
export CDLMD_LICENSE_FILE="1999@127.0.0.1"

module load CCEnv
module load StdEnv/2018.3
module load starccm/13.06.012-R8

slurm_hl2hl.py --format STAR-CCM+ > $SLURM_SUBMIT_DIR/machinefile_$SLURM_JOB_ID

NCORE=$((SLURM_NTASKS * SLURM_CPUS_PER_TASK))

# Workaround for license failures
# Try up to 5 times to get starccm+ to start by checking exit status (throws 143 when fails, 0 when works).
i=1
RET=-1
while [ $i -le 5 ] && [ $RET -ne 0 ]; do
        [ $i -eq 1 ] || sleep 5
        echo "Attempt number: "$i
        starccm+ -power -np $NCORE -podkey $LM_PROJECT   -machinefile $SLURM_SUBMIT_DIR/machinefile_$SLURM_JOB_ID -batch $SLURM_SUBMIT_DIR/your-simulation-file.java $SLURM_SUBMIT_DIR/your-simulation-file.sim > $SLURM_JOB_ID.results
        RET=$?
        i=$((i+1))
   done
exit $RET

License debugging

After establishing the ssh tunnel, you may check that you can reach the server from the compute/debug node with:

$ telnet localhost 1999

If there is no response there is a problem with the tunnel or the server at flex.cd-adapco.com

You may debug the license acquisition process on a compute/debug node (you may use on your script as well) by issuing the following:

$ lmutil lmstat -c 1999@localhost

To check all the licenses available you may use:

$ lmutil lmstat -c 1999@localhost -a

To check the license server from a login node or a datamover your may use:

lmutil lmstat -c 1999@flex.cd-adapco.com

If you see the except below on the output of your run, you need to arrange with your PI to renew the podkey/license for your group

Asked for 1 license of ccmppower and got 0
Asked for 1 license of ccmppowerplus and got 0
PoD database message: -20002 *FAILED_TOKEN_EXPIRED*

If you are using your own license server, you'll need to ensure your firewall is open to income star-ccm+ license requests coming from the following IPs:

142.150.188.70
142.150.188.71
142.150.188.72
142.150.188.73
142.150.188.75
142.150.188.76
142.150.188.77
142.1.174.227
142.1.174.228

STAR-CCM+

Contents