HPC doc.



On Baobab, many applications are already installed. Not every application is listed below. To get a complete list, you need to use Module - lmod when logged. Example to have a complete list:

module spider


You can use SCM ADF on one ore more nodes of the cluster. Please note that you must use “module” (see Module - lmod) in you sbatch script to set the variables correctly.

Please see ADF example on GitLab for some example and scripts related to ADF.


Do not launch ADF using srun. ADF is a wrapper which uses srun internally.


ADF needs a fast local scratch space. On Baobab, the local scratch of each node is only about 180GB. If you need more space, you need to find another solution (do the calculation on more nodes, do not use local scratch, buy us new hard disks)


By default, the local scratch space is defined in the SCM_TMPDIR environment variable, /scratch as set by module load ADF . This default value will gives error when calling ADFview on the login nodes, given that /scratch does not exist there. You can overcome it by using the Linux default /tmp :

capello@login2:~$ module load ADF/2019.104
capello@login2:~$ SCM_TMPDIR=/tmp adfview

Ansys Fluent

You can use Fluent on Baobab. To do so, please see Fluent on gitlab and adapt the sbatch script.


You can use Fall3D on Baobab. To do so, please see Fall3d on gitlab and adapt the sbatch script.


You can use Gaussian g09 on one node of the cluster. Please note that you must use “module” (see Module - lmod) in you sbatch script to set the variables correctly. When using the module, it will set the variable GAUSS_SCRDIR to /scratch of the local hard disk of the allocated node. This should lower the calculation time and as well lower the usage of the shared filesystem. See below for other optimizations.

There are two versions of g09 on the cluster. Revision c01 and d01.

Please see Gaussian example on GitLab for some examples and scripts related to Gaussian.

To optimize the run, you can add some lines in your job file. If you need more than 190 GB of scratch space, you should add the line (adapt /home/yourusername to your own path):


You may as well specify how much memory you want to use. By default, Gaussian will use 250MB of ram. You can try with 50GB for example:


You need to specify as well how many CPU cores you want to use:



To use git on the cluster you need to do the following:

Add that to your ${HOME}/.gitconfig:

createObject = rename

To invoke git:

git clone --no-hardlinks


Usage :



NB , the following instructions come from <>.

We provide an internal Gurobi token server, here how to use it:

capello@login2:~$ module load Gurobi
capello@login2:~$ gurobi_cl --tokens

Checking status of Gurobi token server 'master.cluster'...

Token server functioning normally.
Maximum allowed uses: 4096, current: 0



Please see Keras on gitlab for some examples and scripts related to Keras


If you would like to use a different license server (by default, you can specify its URL in the ${HOME}/.Mathematica/Licensing/mathpass file, prependend by an exclamation mark:

capello@login2:~$ mkdir -p ~/.Mathematica/Licensing/
capello@login2:~$ cat <<EOF >~/.Mathematica/Licensing/mathpass
capello@login2:~$ cat ${HOME}/.Mathematica/Licensing/mathpass

See and for more information.


Matlab is available in Baobab in different versions:

$ module spider matlab


Keep in mind that it’s a licensed program, and that the licenses are shared with the whole university. To be fair with the other users, we have set up a limitation on the number of licenses you can use. We kindly ask you to specify in your sbatch file that you are using Matlab in order to keep the limitation effective. If you are using some licensed toolbox, like Wavelet_Toolbox, you need to specify it as well. If you don’t, your job may be killed without further notice in case we are out of licenses.

Example to specify that you need Matlab:


Example to specify that you need the Wavelet_Toolbox:


See the licenses available on Baobab:

scontrol show lic

If you need a license not listed here, please ask us at


You need to specify at LEAST the Matlab license, and zero or more toolbox.

To run Matlab in batch mode, you can create a batch file like this one:


#SBATCH --cpus-per-task=1
#SBATCH --ntasks=1

module load matlab



echo "Running ${BASE_MFILE_NAME}.m on $(hostname)"

srun matlab -nodesktop -nosplash -nodisplay -r ${BASE_MFILE_NAME}

In this example, you need to have your code in the file hello.m.

You submit the Matlab job like a normal sbatch SLURM job:

sbatch ./yourBatch

Parallel with Matlab

Since version 2014, you are not anymore limited to 12 cpu cores.

Please see Matlab on gitlab for some examples and scripts related to Matlab parallel

As we are talking about parallel and not distributed Matlab, you can consided Matlab as a multithread application.

see how to submit a multithread job Multithreaded jobs


If you are a parallel or distributed Matlab specialist and you have some hints, you are very welcome to contact us!

Pass sbatch arguments to Matlab

You can pass arguments from sbatch to Matlab as described below.

Example of sbatch file (

[sbatch part as usual]



module load matlab

#the variable you want to pass to matlab

echo "Starting at $(date)"
echo "Running ${MATLAB_MFILE} on $(hostname)"

# we call the matlab function (see the parenthesis around the argument) and the argument type will be integer.
srun matlab -nodesktop -nosplash -nodisplay -r "${BASE_MFILE_NAME}($job_array_index)"
echo "Finished at $(date)"

Example of Matlab file (test.m)

function test(job_array_index)

fprintf('array index: %d\n', job_array_index)

See arguments with Matlab on gitlab for more examples.

Compile your Matlab code

Thanks to Philippe Esling for his contribution to this procedure.

The idea of compiling Matlab code is to save on licenses usage. Indeed, once compiled, a Matlab code can be run without using any license.

The Matlab compiler is named MCC

First load the needed modules:

module load foss/2016a matlab/2016b

Let’s say you want to compile this .m file:

function hello(name)
   strcat({'Hello '}, name)

This operation compiles it (this takes some time) :

DISPLAY="" mcc -m -v -R '-nojvm, -nodisplay' -o hello hello.m

If you have some other .m files that you need to include, you need to explicitly specify their location as follows:

mcc -m -v -R '-nojvm, -nodisplay' -I /path/to/functions/ -I /path/to/other/functions/ [...]

The resulting files are a text file named readme.txt, a script named and an executable named hello.

You can then launch the executable hello like any other executable using a sbatch script:


#SBATCH --partition=debug-EL7
#SBATCH --ntasks=1

module load foss/2016a matlab/2016b

srun ./hello Philippe

In this case, Philippe is an argument for the function hello. Be careful, arguments are always passed to Matlab as strings.


You do NOT need to specify the Matlab license once compiled, and you are not restricted by the number of available licenses.

Please see compile Matlab on gitlab for some examples and scripts related to Matlab compilation.

Matlab PATH

If you need to add a directory to the Matlab path, for example to use a toolbox installed by you, please proceed as follow.

Add the needed path recursively to matlab path:


Save the current matlab path to the default matlab definition path:


You can now access any file from the toolbox directly:


Matlab java.opts

If you are getting erros such as the one listed here when using Matlab through X2go, you can try the following changes to mitigate the issue.

Create a file named java.opts in the default startup folder which is the folder from which you started MATLAB.

Put the following content in it:


And restart Matlab. This will probably reduce the graphical performance but should not impact the computation time.


To use the Wavelab library with Matlab, load Matlab 2014.

module load matlab/2014b

Launch Matlab as usual and type this command to go to the Wavelab library:

cd /opt/wavelab/Wavelab850/
Wavepath (answer /opt/wavelab/Wavelab850)


See Meep on gitlab for an example how to use Meep.

Nvidia CUDA

Currently on Baobab there are several nodes equiped with GPUs. To request a GPU, it’s not enough the specify a partition with nodes having GPUs, you must as well specify how many GPUs and optionnaly the needed type.

To specify how many GPU you request, use the option --gres=gpu:n with n having a value between 1 and the maximum according to the table below.

You should also specify the type of GPU you want:

  • titan, for single precision computation, like machine learning.
  • pascal, for double precision computation like physical simulations.
  • rtx, to accelerate machine learning, data science workflows and ray tracing.

Example to request three titan cards: --gres=gpu:titan:3.

In the following table you can see which type of GPU is available on Baobab.

GPU model Compute Capability SLURM resource numbers per node node partition
titan x (pascal) 6.1 titan 3 gpu[002] shared-gpu, dpnc-gpu
p100 6.0 pascal 5 gpu[004-005] shared-gpu, dpt-gpu, kruse-gpu
p100 6.0 pascal 8 gpu[006] shared-gpu, kruse-gpu, hepia-gpu
p100 6.0 pascal 4 gpu[007] shared-gpu, schaer-gpu
titan x (pascal) 6.1 titan 8 gpu[008] shared-gpu, kalousis-gpu
titan x (pascal) 6.1 titan 8 gpu[009-010] shared-gpu, cui-gpu
RTX 2080 Ti (turing) 7.5 rtx 2 gpu[011-014] shared-gpu, hepia-gpu, gervasio-gpu


  • shared-gpu with a max execution time of 12h, tesla, titan and pascal cards
  • dpt-gpu with a max execution time of 2 days, 5 x pascal GPU cards
  • dpnc-gpu with a max execution time of 2 days, 6 x titan GPU cards
  • kruse-gpu with a max execution time of 2 days, 10 x pascal GPU cards
  • hepia-gpu with a max execution time of 2 days, 3 x pascal GPU cards
  • schaer-gpu with a max execution time of 2 days, 4 x pascal GPU cards
  • kalousis-gpu with a max execution time of 7 days, 8 x titan GPU cards
  • cui-gpu with a max execution time of 2 days, 16 x titan GPU cards
  • hepia-gpu with a max execution time of 2 days, 2 x rtx GPU cards

Example of script (see also ):

#!/bin/env bash

#SBATCH --partition=shared-gpu
#SBATCH --time=01:00:00
#SBATCH --gres=gpu:titan:1

module load CUDA

# see here for more samples:
# /opt/cudasample/NVIDIA_CUDA-8.0_Samples/bin/x86_64/linux/release/

# if you need to know the allocated CUDA device, you can obtain it here:

srun deviceQuery

If you want to see what GPUs are in use in a given node:

scontrol -d show node gpu002

In this case, this means that node gpu002 has three Titan cards, and all of them are allocated.

If you just need a gpu and you don’t care of the type, don’t specify it:

#SBATCH --gres=gpu:1

It’s not possible to put two types in the GRES request, but you can ask for specific compute capability, for example you don’t want a compute capability smaller than 6.0:

#SBATCH --gres=gpu:1






You can use OpenCL on CPU. To compile your software, please proceed as following:

gcc  -I/opt/intel/opencl-1.2-sdk- -L/opt/intel/opencl-1.2-sdk- -Wl,-rpath,/opt/intel/opencl/lib64/ -lOpenCL -o hello hello.c





Download the latest palabos version on

tar xzf palabos-v1.3r0.tgz
cd palabos-v1.3r0/examples/benchmarks/cavity3d

If you want to use more cores to compile it, edit the Makefile of your project and change the line

Cons     = $(palabosRoot)/scons/ -j 2 -f $(palabosRoot)/SConstruct

Replace the number 2 by the number of cores that you have at your disposal.

For example if you want to use a full 16 core node to do that, you can compile Palabos like this:

srun --cpus-per-task 16 --ntasks=1 --exclusive make

Distant Paraview

Thanks to Orestis for this tutorial.


  1. Do not use at any point X11 forwarding it will be done by paraview itself.
  2. You must have the SAME version of paraview on your local machine and on baobab (5.3.0).

Baobab connection:


Get some ressources in interactive mode (4 cores here, you can add a lot more options here if you want). You can even ask for gpu nodes (more on this afterwards):

salloc -n 4

Determine on which node your ressources are affacted (here node001):



On another terminal (in your pc) open an ssh tunnel to your NODE (in this case node001):

ssh -L 11150:node001:11111

On the first terminal load paraview (if not loaded by default) and launch pserver:

module load foss/2016b ParaView/5.3.0-mpi
srun pvserver --server-port=11111

If you asked for GPU nodes the command is slightly different:

srun pvserver --server-port=11111 -display :0.0 --use-offscreen-rendering

On your local machine launch paraview and click on Connect. There you should find the menu to add a server. Put the name you want in Name, leave Client/Server as Server Type, and localhost in Host. The only very important configuration is the port which should be 11150 (the same number as in 11150:node001 from before). Save the configuration (click on Configure and Save, leave startup as Manual) and the click on Connect. The remote paraview session should start immedtiatly. There should be an error message: Display is not accessible on the server side. Remote rendering will be disabled. This message is normal.


Default Python version on the cluster is Python 2.7.5.

You can have access to modern Python through Module - lmod

The Python version provided by module come with a lot of packages already installed.

You can check with module spider what are the packages provided. Example:

ml spider Python/3.7.4
Included extensions
alabaster-0.7.12, asn1crypto-0.24.0, atomicwrites-1.3.0, attrs-19.1.0,
Babel-2.7.0, bcrypt-3.1.7, bitstring-3.1.6, blist-1.3.6, certifi-2019.9.11,
cffi-1.12.3, chardet-3.0.4, Click-7.0, cryptography-2.7, Cython-0.29.13,
deap-1.3.0, decorator-4.4.0, docopt-0.6.2, docutils-0.15.2, ecdsa-0.13.2,
future-0.17.1, idna-2.8, imagesize-1.1.0, importlib_metadata-0.22,
ipaddress-1.0.22, Jinja2-2.10.1, joblib-0.13.2, liac-arff-2.4.0,
MarkupSafe-1.1.1, mock-3.0.5, more-itertools-7.2.0, netaddr-0.7.19,
netifaces-0.10.9, nose-1.3.7, packaging-19.1, paramiko-2.6.0, pathlib2-2.3.4,
paycheck-1.0.2, pbr-5.4.3, pip-19.2.3, pluggy-0.13.0, psutil-5.6.3, py-1.8.0,
py_expression_eval-0.3.9, pyasn1-0.4.7, pycparser-2.19, pycrypto-2.6.1,
Pygments-2.4.2, PyNaCl-1.3.0, pyparsing-2.4.2, pytest-5.1.2, python-
dateutil-2.8.0, pytz-2019.2, requests-2.22.0, scandir-1.10.0,
setuptools-41.2.0, setuptools_scm-3.3.3, six-1.12.0, snowballstemmer-1.9.1,
Sphinx-2.2.0, sphinxcontrib-applehelp-1.0.1, sphinxcontrib-devhelp-1.0.1,
sphinxcontrib-htmlhelp-1.0.2, sphinxcontrib-jsmath-1.0.1, sphinxcontrib-
qthelp-1.0.2, sphinxcontrib-serializinghtml-1.1.3, sphinxcontrib-
websupport-1.1.2, tabulate-0.8.3, ujson-1.35, urllib3-1.25.3,
virtualenv-16.7.5, wcwidth-0.1.7, wheel-0.33.6, xlrd-1.2.0, zipp-0.6.0

If you need numpy, SciPy, pandas, mpi4py, they are provided by the SciPy-bundle module.

Example to load a recent version of Python with SciPy:

ml GCC/8.2.0-2.31.1 OpenMPI/3.1.3 Python/3.7.2 SciPy-bundle/2019.03

Custom Python lib

If you need to install a python library or a different version of the ones already installed, virtualenv is the solution.

Python-virtualenv is installed on Baobab

Begin by loading a version of python using module (see above)

Create a new virtualenv if it’s not already existing (put it where you want and name it like you want):

virtualenv --no-site-packages ~/baobab_python_env

This will create a directory named baobab_python_env in your home directory.

Everytime you want to use your virtualenv, you should activate it first:

. ~/baobab_python_env/bin/activate

Install all the needed packages in the environment:

~/baobab_python_env/bin/pip install mpi4py

Use your new environment:


R project and RStudio

The latest version of R (if it’s not the latest, you can ask us to install it) is installed on the cluster.

Please see here for an sbatch example with R. You will find as well an exemple using the R package parallel.


RStudio is IDE (Integrated Development Environment) for R, basically a more user friendly version than plain R.

With the Baobab upgrade to CentOS 7 (cf. ) we do not provide anymore a central RStudio.

Instead, you can download the upstream Open Source binary RStudio Desktop version (cf. ) and directly use it, here the instructions:

  1. install it in your ${HOME} folder:
capello@login2:~$ mkdir Downloads
capello@login2:~$ cd Downloads
capello@login2:~/Downloads$ wget ${URL_FOR_rstudio-${VERSION}-x86_64-fedora.tar.gz}
capello@login2:~/Downloads$ tar axvf rstudio-${VERSION}-x86_64-fedora.tar.gz
  1. launch an interactive session:
    1. connect to the cluster using ssh -Y from a machine with an X server (see Graphical access).
    2. load one of the R version supported by RStudio, for example:
capello@login2:~$ module spider R/3.6.0

  R: R/3.6.0
      R is a free software environment for statistical computing and

    You will need to load all module(s) on any one of the lines below
    before the "R/3.6.0" module is available to load.

      GCC/8.2.0-2.31.1  OpenMPI/3.1.3
capello@login2:~$ module load GCC/8.2.0-2.31.1  OpenMPI/3.1.3
capello@login2:~$ module load R/3.6.0
  1. run RStudio:
capello@login2:~$ srun -p debug-EL7 -n 1 -c 16 ~/Downloads/rstudio-${VERSION}/bin/rstudio
Doing so, you will have 16 cores on one node of the partition debug-EL7 for a max time of 15 minutes. Specify the appropriate duration time, partition, etc. like you would do for a normal job.

R packages

You can install R packages as a user. Just follow once the steps given below:

Create a file named .Rprofile (note the dot in front of the file) in your home directory with the following content:

cat(".Rprofile: Setting Switzerland repository\n")

r = getOption("repos") # hard code the Switzerland repo for CRAN
r["CRAN"] = ""
options(repos = r)

Create a file named .Renviron (note the dot in front of the file) in your home directory with the following content:


Create a directory where to store the installed R packages:

mkdir ~/Rpackages

Once done, make sure you have loaded R with module. Then, from a R command line you can install a R package.

For R version 3.5 and below :


For R version 3.6 or above :


Use your newly installed package:



Singularity is a Docker like for HPC. It’s installed on Baobab and available through module.

module load GCC/6.3.0-2.27 Singularity/2.4.2

As you don’t have root access on Baobab, you cannot build recipe file on Baobab. If you need to do this, you may want to build the image on your own machine and transfer the image to Baobab.

You can download existing images from shub (singularity hub) or from docker (docker public or private registry).

The image will be converted to a readonly squashfs on Baobab disk. It’s not possible to have writable images as user. Instead you should build the image as sandbox.

Example to download an existing docker image and build a squashf image from it:

singularity build lolcow.simg docker://godlovedc/lolcow

Then you will end-up with a singularity image named lolcow.simg.

Singularity images are immutable.

Once you have a singularity image, you can do various things:

  • run to run a container (the script named singularity at the root of the image, proceed as follow:

    singularity run lolcow.simg

When you import a docker image, the ENTRYPOINT is used to create the singularity run script.

  • exec to execute a single command inside the container.
  • shell to launch a container and to spawn a shell (/bin/bash)

Download an image from a custom registry:

singularity build docker://


Stata version 13 mp 16 and version 14 mp 24 are available on the cluster.

To use it, you need to add the stata path to your PATH:

module load stata/13mp16


module load stata/14mp24

The Stata binaries are stata-mp or xstata-mp for the graphical interface.

If you need a graphical interactive session, please proceed as follows:

salloc -n1 -c 16 --partition=debug-EL7 --time=15:00 srun -n1 -N1 --pty $SHELL

Doing so will launch a graphical Stata on a debug node with 16 cores for a 15 minutes session. See on this document for other partition/time limits.


Please keep in mind that the cluster may be full and that you will have to wait until the resources are allocated to you. It’s best to launch stata in batch mode.

To launch stata in batch mode, see Multithreaded jobs and specify that you want one task and n cores.

Please see here for an sbatch example with stata.


Please see TensorFlow on gitlab for some examples and scripts related to TensorFlow.

ATTENTION , the module:TensorFlow we provide are compiled with GPU support (see ), thus they must be used on a GPU partition (cf. Nvidia CUDA ).


Please see Theano on gitlab for some examples and scripts related to Theano