Skip to content

Installing and configuring the EESSI test suite

This page covers the requirements, installation and configuration of the EESSI test suite.

Requirements

The EESSI test suite requires

Installing Reframe

General instructions for installing ReFrame are available in the ReFrame documentation. To check if ReFrame is available, run the reframe command:

reframe --version
(for more details on the ReFrame version requirement, click here)

Two important bugs were resolved in ReFrame's CPU autodetect functionality in version 4.3.3.

We strongly recommend you use ReFrame >= 4.3.3.

If you are using an older version of ReFrame, you may encounter some issues:

  • ReFrame will try to use the parallel launcher command configured for each partition (e.g. mpirun) when doing the remote autodetect. If there is no system-version of mpirun available, that will fail (see ReFrame issue #2926).
  • CPU autodetection only worked when using a clone of the ReFrame repository, not when it was installed with pip or EasyBuild (as is also the case for the ReFrame shipped with EESSI) (see ReFrame issue #2914).

Installing ReFrame test library (hpctestlib)

The EESSI test suite requires that the ReFrame test library (hpctestlib) is available, which is currently not included in a standard installation of ReFrame.

We recommend installing ReFrame using EasyBuild (version 4.8.1, or newer), or using a ReFrame installation that is available in the EESSI repository (version 2023.06, or newer).

For example (using EESSI):

source /cvmfs/software.eessi.io/versions/2023.06/init/bash
module load ReFrame/4.3.3

To check whether the ReFrame test library is available, try importing a submodule of the hpctestlib Python package:

python3 -c 'import hpctestlib.sciapps.gromacs'

Installation

To install the EESSI test suite, you can either use pip or clone the GitHub repository directly:

Using pip

pip install git+https://github.com/EESSI/test-suite.git

Cloning the repository

git clone https://github.com/EESSI/test-suite $HOME/EESSI-test-suite
cd EESSI-test-suite
export PYTHONPATH=$PWD:$PYTHONPATH

Verify installation

To check whether the EESSI test suite installed correctly, try importing the eessi.testsuite Python package:

python3 -c 'import eessi.testsuite'

Configuration

Before you can run the EESSI test suite, you need to create a configuration file for ReFrame that is specific to the system on which the tests will be run.

Example configuration files are available in the config subdirectory of the EESSI/test-suite GitHub repository](https://github.com/EESSI/test-suite/tree/main/config), which you can use as a template to create your own.

Configuring ReFrame environment variables

We recommend setting a couple of $RFM_* environment variables to configure ReFrame, to avoid needing to include particular options to the reframe command over and over again.

ReFrame configuration file ($RFM_CONFIG_FILES)

(see also RFM_CONFIG_FILES in ReFrame docs)

Define the $RFM_CONFIG_FILES environment variable to instruct ReFrame which configuration file to use, for example:

export RFM_CONFIG_FILES=$HOME/EESSI-test-suite/config/example.py

Alternatively, you can use the --config-file (or -C) reframe option.

See the section on the ReFrame configuration file for more information.

Search path for tests ($RFM_CHECK_SEARCH_PATH)

(see also RFM_CHECK_SEARCH_PATH in ReFrame docs)

Define the $RFM_CHECK_SEARCH_PATH environment variable to tell ReFrame which directory to search for tests.

In addition, define $RFM_CHECK_SEARCH_RECURSIVE to ensure that ReFrame searches $RFM_CHECK_SEARCH_PATH recursively (i.e. so that also tests in subdirectories are found).

For example:

export RFM_CHECK_SEARCH_PATH=$HOME/EESSI-test-suite/eessi/testsuite/tests
export RFM_CHECK_SEARCH_RECURSIVE=1

Alternatively, you can use the --checkpath (or -c) and --recursive (or -R) reframe options.

ReFrame prefix ($RFM_PREFIX)

(see also RFM_PREFIX in ReFrame docs)

Define the $RFM_PREFIX environment variable to tell ReFrame where to store the files it produces. E.g.

export RFM_PREFIX=$HOME/reframe_runs

This involves:

  • test output directories (which contain e.g. the job script, stderr and stdout for each of the test jobs)
  • staging directories (unless otherwise specified by staging, see below);
  • performance logs;

Note that the default is for ReFrame to use the current directory as prefix. We recommend setting a prefix so that logs are not scattered around and nicely appended for each run.

If our common logging configuration is used, the regular ReFrame log file will also end up in the location specified by $RFM_PREFIX.

Warning

Using the --prefix option in your reframe command is not equivalent to setting $RFM_PREFIX, since our common logging configuration only picks up on the $RFM_PREFIX environment variable to determine the location for the ReFrame log file.