Special applications, and tips & tricks

Many different applications are used by scientists from the various fields on the HPC systems at RRZE. The HPC team at RRZE does not have any experience with most of the applications. Nevertheless, we try to collect some useful information and tips&tricks for key applications on the following pages. The pages will not teach you how to use the applications but collect information which is specific to the usage on the HPC systems of RRZE.

Central installation of software

As the parallel computers of RRZE are operated diskless, all system software has to reside in a RAM disk, i.e. in main memory. Therefore, only limited packages from the Linux distribution can be installed on the compute nodes, and the compute nodes only contain a subset of the packages installed on the login nodes. Most (application) software but also libraries therefore will be installed in /apps and made available as modules. Multiple versions of a single software can be provided in that way, too.

As a general rule: software will be installed centrally

    1. if there are multiple groups which benefit from the software, or
    2. if the software is very easy to install.

In both cases RRZE will more and more request that a least one group acts as mentor for the software, i.e. provides RRZE with simple input for validation of the installation, but moreover, that group has to provide limited support for other groups to get started with the software.

Notes on specific software

Molecular dynamics for chemistry, life science, and material science

  • Amber & AmberTools – suite of biomolecular simulation programs. Here, the term „Amber“ does not refer to the set of molecular mechanical force fields for the simulation of biomolecules but to the package of molecular simulation programs consisting of the AmberTools (sander and many more) and Amber (pmemd).
  • DL_POLY / DL_POLY_CLASSIC – general-purpose classical molecular dynamics simulation software.
    DL_POLY with its fine grain IO can easily kill $FASTTMP. RRZE will kill / throttle any jobs which abuse the parallel file system.
  • GROMACS – versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. It is primarily designed for biochemical molecules like proteins, lipids and nucleic acids that have a lot of complicated bonded interactions, but since GROMACS is extremely fast at calculating the non-bonded interactions (that usually dominate simulations) many groups are also using it for research on non-biological systems, e.g. polymers.
  • IMD – software package for classical molecular dynamics simulations. Several types of interactions are supported and a rich choice of simulation options is available. There is no restriction on the number of particle types.
  • LAMMPS – classical molecular dynamics code with a focus on materials modeling. LAMMPS has potentials for solid-state materials (metals, semiconductors) and soft matter (biomolecules, polymers) and coarse-grained or mesoscopic systems. It can be used to model atoms or, more generically, as a parallel particle simulator at the atomic, meso, or continuum scale.

Quantum chemistry

  • CPMD – parallelized plane wave / pseudopotential implementation of Density Functional Theory, particularly designed for ab-initio molecular dynamics.
  • Molpro – a comprehensive system of ab initio programs for advanced molecular electronic structure calculations.
  • OpenMolcas – a quantum chemistry software package using the multiconfigurational approach to the electronic structure.
  • ORCA – an ab initio quantum chemistry program package that contains modern electronic structure methods including density functional theory, many-body perturbation, coupled cluster, multireference methods, and semi-empirical quantum chemistry methods. Its main field of application is larger molecules, transition metal complexes, and their spectroscopic properties.
  • Quantum Espresso – integrated suite of open-source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials.
  • TURBOMOLE program package for ab initio electronic structure calculations.
  • VASP – Vienna Ab initio Simulation Package (VASP) is a computer program for atomic scale materials modeling, e.g. electronic structure calculations and quantum-mechanical molecular dynamics, from first principles.
    Not yet installed centrally owing to license restrictions – RRZE is waiting for the approval of its „data center installation and maintenance license“.

Computational fluid dynamics (CFD), multiphysics and FE, climatology and glaciology

  • ANSYS CFX – commercial CFD software tool with a special focus on turbomachinery, such as pumps, fans, compressors and gas and hydraulic turbines.
  • ANSYS Fluent – commercial CFD to model flow, turbulence, heat transfer and reactions in all sorts of applications.
  • ANSYS Mechanical – commercial finite element analysis (FEA) tool, to solve complex mechanical problems.
  • Simcenter STAR-CCM+ – commercial software tool for CFD and more generally computational continuum mechanics (by CD-adapco or nowadays Siemens PLM).
  • OpenFOAM.org/OpenFOAM.com – leading free, open-source software for computational fluid dynamics but also an extensive C++ toolbox for the development of customized numerical solvers, and pre-/post-processing utilities for the solution of continuum mechanics problems.
  • Abaqus – software suite for finite element analysis and computer-aided engineering.
    Currently no (known) active users on the HPC systems.
  • COMSOL Multiphysics – general-purpose simulation software for modeling designs, devices, and processes in all fields of engineering, manufacturing, and scientific research.
  • Elmer/Ice – open-source Finite Element software for ice sheet, glaciers, and ice flow modeling.
    Avoid MATC expressions in the SIF input.
  • WRF – Weather Research and Forecasting (WRF) is a next-generation mesoscale numerical weather prediction system designed for both atmospheric research and operational forecasting applications.

Mathematical and statistical software

  • Matlab – numerical computing environment and proprietary programming language developed by MathWorks
  • R – „Microsoft R Open“ is installed on Woody. It is an enhanced distribution of R.
    Additional packages available from MRAN; can be installed centrally upon request.

Machine learning and big data

  • cuDNN in version 7 is installed on TinyGPU and the GPU nodes of Emmy since autumn 2019 as Nvidia relaxed the license conditions.
    There is no need to load a module as the include files and libraries are in /usr/include and /usr/lib[64].
  • TensorFlow & co
    RRZE currently does not provide any central installation for two three reasons: (a) the software changes very rapidly; (b) NVIDIA imposes very strict conditions on cuDNN – you can only get a personal license; (c) no group volunteered as mentor so far.
  • OpenPosemulti-person gesture detection (i.e. human body, hand, facial, and foot keypoints)
    A detailed description on how to get OpenPose installed on RRZE’s HPC systems can be found in http://peter-uhrig.de/openpose-with-nvcaffe-in-a-singularity-container-with-support-for-multiple-architectures/


  • software for the astrophysics community is maintained by ECAP.

Virtualization and containers

  • Singularity in version 3.x is available on all HPC systems. Thus, you can run your own environment in a docker-like fashion.
    Note: When importing containers e.g. from dockerhub set the environment variable SINGULARITY_CACHEDIR to save space in $HOME as it defaults to ~/.singularity/docker/ and ~/.singularity/metadata/.

HPC tools

  • DDT – parallel debugger.
  • TotalView – parallel debugger. See in HPC-Wiki.
  • LIKWID – tool suite for performance-oriented programmers and administrators. The term LIKWID stands for ‚Like I know what I do‘. See in HPC-Wiki and on Github .
  • Intel Trace Collector / Analyzer – tool for profiling and checking the correctness of MPI communication. A short how-to is available here or in the HPC-Wiki.
  • Intel Vtune & friends (advisor, inspector, performance_snapshots, vtune_amplifier) – tools for detailed performance profiling. See HPC-Wiki for Vtune and Intel Advisor.
    Only available on special machines – contact support-hpc@fau.de for details.