Difference between revisions of "Visualization"

From SciNet Users Documentation
Jump to: navigation, search
(Manual Niagara Host Configuration)
(VisIt Client-Server Configuration)
 
(28 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Visualization =
 
 
== Software Available ==
 
== Software Available ==
We have installed the latest versions of the open source visualization suites: VMD, VisIt and ParaView.
+
We have recent versions of the open source visualization suites installed on Niagara: VMD (1.9.4) is available as a module in the default NiaEnv/'''2019b''' environment. The NiaEnv/'''2018a''' environment additionally has modules for VisIt (2.13.1) and ParaView (5.5.0, and 5.6.0 for offscreen rendering only).
  
 
Notice that for using ParaView you need to explicitly specify one of the mesa flags in order to avoid trying to use openGL, i.e.,
 
Notice that for using ParaView you need to explicitly specify one of the mesa flags in order to avoid trying to use openGL, i.e.,
Line 8: Line 7:
 
   paraview --mesa-swr
 
   paraview --mesa-swr
  
Notice that Niagara does not have specialized nodes nor specially designated hardware for visualization, so if you want to perform interactive visualization or exploration of your data you will need to submit an interactive job (debug job, see [[#Testing]]).
+
Notice that Niagara does not have specialized nodes nor specially designated hardware for visualization, so if you want to perform interactive visualization or exploration of your data you will need to submit an interactive job (debug job, see [[https://docs.scinet.utoronto.ca/index.php/Niagara_Quickstart#Testing]]).
 
For the same reason you won't be able to request or use GPUs for rendering as there are none!
 
For the same reason you won't be able to request or use GPUs for rendering as there are none!
  
Line 16: Line 15:
 
For performing an interactive visualization session in this way please follow these steps:
 
For performing an interactive visualization session in this way please follow these steps:
 
<ol>
 
<ol>
<li> ssh into niagara.scinet.utoronto.ca with the -X/-Y flag for x-forwarding
+
<li> ssh into niagara.scinet.utoronto.ca with the -X/-Y flag for [https://docs.scinet.utoronto.ca/index.php/SSH#X11_Forwarding X-forwarding]
  
 
<li> request an interactive job, ie.</li>
 
<li> request an interactive job, ie.</li>
Line 24: Line 23:
 
<li> run your favourite visualization program, eg. VisIt/ParaView </li>
 
<li> run your favourite visualization program, eg. VisIt/ParaView </li>
  
   module load visit
+
   module load NiaEnv/2018a visit
 
   visit
 
   visit
  
   module load paraview
+
   module load NiaEnv/2018a paraview
 
   paraview --mesa-swr
 
   paraview --mesa-swr
  
 
<li> exit the debug session.
 
<li> exit the debug session.
 
</ol>
 
</ol>
 +
 +
 +
== Parallel Visualization with VisIt ==
 +
In order to be able to use VisIt's parallel rendering capabilities, the following command must be issued after loading the corresponding VisIt module:
 +
 +
  export  LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${SCINET_VISIT_ROOT}/2.13.1/linux-x86_64/lib
 +
 +
In this case, the command is used for the visit/2.13.1 module from the NiaEnv/2018a environment.
 +
  
 
== Remote Visualization -- Client-Server Mode ==
 
== Remote Visualization -- Client-Server Mode ==
Line 49: Line 57:
 
Choose *one* of the methods bellow:
 
Choose *one* of the methods bellow:
  
====Niagara Host Configuration File====
+
==== Niagara Host Configuration File ====
You can just download the Niagara host file, right click on the following link [https://support.scinet.utoronto.ca/~mponce/viz/host_niagara.xml host_niagara.xml] and select save as...  
+
You can just download the Niagara host file, right click on the following link
 +
[https://monitor.scinet.utoronto.ca/gitlab/teaching/ihpcss-2021_sciviz/-/raw/main/examples/host_niagara.xml host_niagara.xml] and select save as...  
 
Depending on the OS you are using on your local machine:
 
Depending on the OS you are using on your local machine:
 
<ul>
 
<ul>
Line 59: Line 68:
 
Restart VisIt and check that the niagara profile should be available in your hosts.
 
Restart VisIt and check that the niagara profile should be available in your hosts.
  
====Manual Niagara Host Configuration====
+
==== Manual Niagara Host Configuration ====
 
If you prefer to set up the verser yourself, instead of the configuration file from the previous section, just follow along these steps.
 
If you prefer to set up the verser yourself, instead of the configuration file from the previous section, just follow along these steps.
 
Open VisIt in your computer, go to the 'Options' menu, and click on "Host profiles..."
 
Open VisIt in your computer, go to the 'Options' menu, and click on "Host profiles..."
Line 95: Line 104:
  
 
Finally, after you are done with these changes, go to the "Options" menu and select "Save settings", so that your changes are saved and available next time you relaunch VisIt.
 
Finally, after you are done with these changes, go to the "Options" menu and select "Save settings", so that your changes are saved and available next time you relaunch VisIt.
 +
 +
 +
==== Multi-node configuration in Client-Server Mode ====
 +
As described in <a href='https://docs.scinet.utoronto.ca/index.php/Visualization#Parallel_Visualization_with_VisIt'>Parallel Visualization</a>, some additional steps must be follow in order to have VisIt running in parallel across multiple nodes even when using the client-server approach.
 +
For that the simplest way to achieve this is to add the following lines to your <code>.bashrc</code> file:
 +
 +
  # load needed modules
 +
  module load NiaEnv/2018a
 +
  module load intel/2018.2 intelmpi/2018.2
 +
  module load visit
 +
  # adjust LD_LIBRARY_PATH for VisIt
 +
  export  LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${SCINET_VISIT_ROOT}/2.13.1/linux-x86_64/lib
  
 
=== ParaView Client-Server Configuration ===
 
=== ParaView Client-Server Configuration ===
Line 104: Line 125:
 
   debugjob
 
   debugjob
  
<li> After getting a compute node, let's say niaXYZW, load the ParaView module and start a ParaView server,</li>
+
<li> After getting a compute node, let's say niaXYZW, load the ParaView "offscreen" module and start a ParaView server,</li>
  
   module load paraview
+
   module load NiaEnv/2018a paraview-offscreen/5.6.0
   pvserver --mesa-swr-ax2
+
   pvserver --mesa-swr-avx2
  
 
The <code>--mesa-swr-avx2</code> flag has been reported to offer faster software rendering using the OpenSWR library.
 
The <code>--mesa-swr-avx2</code> flag has been reported to offer faster software rendering using the OpenSWR library.
Line 138: Line 159:
 
</ol>
 
</ol>
  
====Multiple CPUs====
+
==== Multiple CPUs ====
For performing parallel rendering using multiple CPUs, <code>pvserver</code> should be run using <code>srun</code>, ie. either submit a job script or request a job using
+
For performing parallel rendering using multiple CPUs, <code>pvserver</code> should be run using <code>mpiexec</code>, ie. either submit a job script or request a job using
  
 
<source lang="bash"> salloc --ntasks=N*40 --nodes=N --time=1:00:00</source>
 
<source lang="bash"> salloc --ntasks=N*40 --nodes=N --time=1:00:00</source>
  
   module load paraview
+
   module load NiaEnv/2018a paraview-offscreen/5.6.0
   srun pvserver --mesa
+
   mpiexec pvserver --mesa-swr-avx2
  
 
=== Final Considerations ===
 
=== Final Considerations ===
 
Usually both VisIt and ParaView require to use the same version between the local client and the remote host, please try to stick to that to avoid having incompatibility issues, which might result in potential problems during the connections.
 
Usually both VisIt and ParaView require to use the same version between the local client and the remote host, please try to stick to that to avoid having incompatibility issues, which might result in potential problems during the connections.
 +
 +
== Batch mode and Scripting ==
 +
Both, VisIt and ParaView, allow for batch processing using scripting in different languages.
 +
 +
=== VisIt ===
 +
In order to run or launch the Python interpreter for VisIt, you will need to execute the following command:
 +
 +
<code> visit -cli -no-win -norun engine_par </code>
 +
 +
the 'engine_par' flag is needed in order to run the visualization engine in parallel.
 +
 +
 +
In order to execute scripts in batch mode, use the following command:
 +
 +
<code> visit -cli -no-win -norun engine_par  -s scriptName.py</code>
 +
 +
Notice that for running in parallel, in addition to loading the VisIt module, an mpi  module should also be loaded.
 +
Eg.
 +
 +
  module load NiaEnv/2018a
 +
  module load visit
 +
  module load intel/2018.2 intelmpi/2018.2
 +
 +
 +
 +
Alternatively, if one wants to assign resources from the cluster just from within the script one should add the following line to the script
 +
 +
  OpenComputeEngine("localhost", ("-l", "srun", "-np", "40"))
 +
 +
 +
References:
 +
<br>
 +
[https://www.visitusers.org/index.php?title=ParallelPorting#Making_sure_you_actually_have_a_parallel_engine https://www.visitusers.org/index.php?title=ParallelPorting#Making_sure_you_actually_have_a_parallel_engine]
 +
<br>
 +
[https://www.visitusers.org/index.php?title=VisIt-tutorial-Python-scripting https://www.visitusers.org/index.php?title=VisIt-tutorial-Python-scripting]
 +
<br>
 +
[https://www.visitusers.org/index.php?title=Python_examples https://www.visitusers.org/index.php?title=Python_examples]
 +
<br>
 +
[https://www.visitusers.org/index.php?title=VisIt-tutorial-Advanced-scripting https://www.visitusers.org/index.php?title=VisIt-tutorial-Advanced-scripting]
 +
 +
=== ParaView ===
 +
ParaView offers a python interpreter <code>pvbatch</code> to use
 +
 +
pvbatch --mesa-swr-avx2 --force-offscreen-rendering scriptName.py
 +
 +
 +
References:
 +
<br>
 +
[https://www.paraview.org/Wiki/ParaView_and_Batch https://www.paraview.org/Wiki/ParaView_and_Batch]
 +
<br>
 +
[https://www.paraview.org/Wiki/ParaView/Python_Scripting https://www.paraview.org/Wiki/ParaView/Python_Scripting]
  
 
== Other Versions ==
 
== Other Versions ==
 
Alternatively you can try to use the visualization modules available on the CCEnv stack, for doing so just load the CCEnv module and select your favourite visualization module.
 
Alternatively you can try to use the visualization modules available on the CCEnv stack, for doing so just load the CCEnv module and select your favourite visualization module.

Latest revision as of 06:28, 30 July 2021

Software Available

We have recent versions of the open source visualization suites installed on Niagara: VMD (1.9.4) is available as a module in the default NiaEnv/2019b environment. The NiaEnv/2018a environment additionally has modules for VisIt (2.13.1) and ParaView (5.5.0, and 5.6.0 for offscreen rendering only).

Notice that for using ParaView you need to explicitly specify one of the mesa flags in order to avoid trying to use openGL, i.e., after loading the paraview module, use the following command:

 paraview --mesa-swr

Notice that Niagara does not have specialized nodes nor specially designated hardware for visualization, so if you want to perform interactive visualization or exploration of your data you will need to submit an interactive job (debug job, see [[1]]). For the same reason you won't be able to request or use GPUs for rendering as there are none!

Interactive Visualization

Runtime is limited on the login nodes, so you will need to request a testing job in order to have more time for exploring and visualizing your data. Additionally by doing so, you will have access to the 40 cores of each of the nodes requested. For performing an interactive visualization session in this way please follow these steps:

  1. ssh into niagara.scinet.utoronto.ca with the -X/-Y flag for X-forwarding
  2. request an interactive job, ie.
  3.   debugjob
    

    this will connect you to a node, let's say for the argument "niaXYZW"

  4. run your favourite visualization program, eg. VisIt/ParaView
  5.   module load NiaEnv/2018a visit
      visit
    
      module load NiaEnv/2018a paraview
      paraview --mesa-swr
    
  6. exit the debug session.


Parallel Visualization with VisIt

In order to be able to use VisIt's parallel rendering capabilities, the following command must be issued after loading the corresponding VisIt module:

 export  LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${SCINET_VISIT_ROOT}/2.13.1/linux-x86_64/lib

In this case, the command is used for the visit/2.13.1 module from the NiaEnv/2018a environment.


Remote Visualization -- Client-Server Mode

You can use any of the remote visualization protocols supported for both VisIt and ParaView.

Both, VisIt and ParaView, support "remote visualization" protocols. This includes:

  • accessing data remotely, ie. stored on the cluster
  • rendering visualizations using the compute nodes as rendering engines
  • or both

VisIt Client-Server Configuration

For allowing VisIt connect to the Niagara cluster you need to set up a "Host Configuration".

Choose *one* of the methods bellow:

Niagara Host Configuration File

You can just download the Niagara host file, right click on the following link host_niagara.xml and select save as... Depending on the OS you are using on your local machine:

  • on a Linux/Mac OS place this file in ~/.visit/hosts/
  • on a Windows machine, place the file in My Documents\VisIt 2.13.0\hosts\

Restart VisIt and check that the niagara profile should be available in your hosts.

Manual Niagara Host Configuration

If you prefer to set up the verser yourself, instead of the configuration file from the previous section, just follow along these steps. Open VisIt in your computer, go to the 'Options' menu, and click on "Host profiles..." Then click on 'New Host' and select:

Host nickname = niagara
Remote host name = niagara.scinet.utoronto.ca
Username = Enter_Your_OWN_username_HERE
Path to VisIt installation = /scinet/niagara/software/2018a/opt/base/visit/2.13.1

Click on the "Tunnel data connections through SSH", and then hit Apply!

Visit niagara-01.png


Now on the top of the window click on 'Launch Profiles' tab. You will have to create two profiles:

  1. login: for connecting through the login nodes and accessing data
  2. slurm: for using compute nodes as rendering engines

For doing so, click on 'New Profile', set the corresponding profile name, ie. login/slurm. Then click on the Parallel tab and set the "Launch parallel engine"

For the slurm profile, you will need to set the parameters as seen below:


Visit niagara-02.png Visit niagara-03.png


Finally, after you are done with these changes, go to the "Options" menu and select "Save settings", so that your changes are saved and available next time you relaunch VisIt.


Multi-node configuration in Client-Server Mode

As described in <a href='https://docs.scinet.utoronto.ca/index.php/Visualization#Parallel_Visualization_with_VisIt'>Parallel Visualization</a>, some additional steps must be follow in order to have VisIt running in parallel across multiple nodes even when using the client-server approach. For that the simplest way to achieve this is to add the following lines to your .bashrc file:

 # load needed modules
 module load NiaEnv/2018a
 module load intel/2018.2 intelmpi/2018.2 
 module load visit
 # adjust LD_LIBRARY_PATH for VisIt
 export  LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${SCINET_VISIT_ROOT}/2.13.1/linux-x86_64/lib

ParaView Client-Server Configuration

Similarly to VisIt you will need to start a debugjob in order to use a compute node to files and compute resources. Here are the steps to follow:

  1. Launch an interactive job (debugjob) on Niagara,
  2.  debugjob
    
  3. After getting a compute node, let's say niaXYZW, load the ParaView "offscreen" module and start a ParaView server,
  4.  module load NiaEnv/2018a paraview-offscreen/5.6.0
     pvserver --mesa-swr-avx2
    

    The --mesa-swr-avx2 flag has been reported to offer faster software rendering using the OpenSWR library.

  5. Now, you have to wait a few seconds for the server to be ready to accept client connections.
  6.  Waiting for client...
     Connection URL: cs://niaXYZW.scinet.local:11111
     Accepting connection(s): niaXYZW.scinet.local:11111
    
  7. Open a new terminal without closing your debugjob, and ssh into Niagara using the following command,
  8.  ssh YOURusername@niagara.scinet.utoronto.ca -L11111:niaXYZW:11111 -N
    

    this will establish a tunnel mapping the port 11111 in your computer (localhost) to the port 11111 on the Niagara's compute node, niaXYZW, where the ParaView server will be waiting for connections.

  9. Start ParaView on your local computer, go to "File -> Connect" and click on 'Add Server'. You will need to point ParaView to your local port 11111, so you can do something like
  10. name = niagara
    server type = Client/Server
    host = localhost
    port = 11111
    

    then click Configure, select Manual and click Save.

  11. Once the remote server is added to the configuration, simply select the server from the list and click Connect. The first terminal window that read Accepting connection... will now read Client connected.
  12. Open a file in ParaView (it will point you to the remote filesystem) and visualize it as usual.

Multiple CPUs

For performing parallel rendering using multiple CPUs, pvserver should be run using mpiexec, ie. either submit a job script or request a job using

 salloc --ntasks=N*40 --nodes=N --time=1:00:00
 module load NiaEnv/2018a paraview-offscreen/5.6.0
 mpiexec pvserver --mesa-swr-avx2

Final Considerations

Usually both VisIt and ParaView require to use the same version between the local client and the remote host, please try to stick to that to avoid having incompatibility issues, which might result in potential problems during the connections.

Batch mode and Scripting

Both, VisIt and ParaView, allow for batch processing using scripting in different languages.

VisIt

In order to run or launch the Python interpreter for VisIt, you will need to execute the following command:

visit -cli -no-win -norun engine_par

the 'engine_par' flag is needed in order to run the visualization engine in parallel.


In order to execute scripts in batch mode, use the following command:

visit -cli -no-win -norun engine_par -s scriptName.py

Notice that for running in parallel, in addition to loading the VisIt module, an mpi module should also be loaded. Eg.

  module load NiaEnv/2018a
  module load visit
  module load intel/2018.2 intelmpi/2018.2 


Alternatively, if one wants to assign resources from the cluster just from within the script one should add the following line to the script

  OpenComputeEngine("localhost", ("-l", "srun", "-np", "40"))


References:
https://www.visitusers.org/index.php?title=ParallelPorting#Making_sure_you_actually_have_a_parallel_engine
https://www.visitusers.org/index.php?title=VisIt-tutorial-Python-scripting
https://www.visitusers.org/index.php?title=Python_examples
https://www.visitusers.org/index.php?title=VisIt-tutorial-Advanced-scripting

ParaView

ParaView offers a python interpreter pvbatch to use

pvbatch --mesa-swr-avx2 --force-offscreen-rendering scriptName.py


References:
https://www.paraview.org/Wiki/ParaView_and_Batch
https://www.paraview.org/Wiki/ParaView/Python_Scripting

Other Versions

Alternatively you can try to use the visualization modules available on the CCEnv stack, for doing so just load the CCEnv module and select your favourite visualization module.