FUnTiDES: Fast Unstructured Time Dynamic Equation Solver

FUnTiDES is a collection of simplified codes that represent real scientific applications. It serves as a standard tool for evaluating and comparing the performance of various high-performance computing (HPC) systems, particularly those used for scientific simulations.

---

Included Applications

The current implementation includes two proxy applications for solving the 2nd-order acoustic wave equation in 2D and 3D:

• SEM (Spectral Element Method) A benchmark designed to simulate wave propagation using SEM, a Galerkin-based finite element method for solving partial differential equations (PDEs).

• FD (Finite Difference Method) A benchmark that uses finite-difference stencil operators to simulate wave propagation and solve PDEs.

A key feature of these proxy applications is their adaptability to different programming models and HPC architectures. They are also easy to build and run, making them accessible to both researchers and developers.

---

Supported Programming Models

The SEM proxy currently supports:

• Kokkos — for performance portability

Note: Kokkos is included as a Git submodule and will be compiled automatically when enabled.

---

Supported Data Containers

The current SEM proxy supports the following data container:

• std::vector (default for serial )

---

Quick Start: Build and Run

Step 1: Compile and Install

mkdir build
cd build
cmake -DUSE_KOKKOS=ON -DENABLE_CUDA=ON -DUSE_VECTOR=OFF -DENABLE_PYWRAP=OFF ..

Both SEM and FD applications are compiled.

Step 2: Run Tests & Benchmarks

Unit tests only

ctest -LE benchmark

Benchmarks only, results will be stored in results generated in build/Benchmarking as a json file.

ctest -L benchmark

Or just both

ctest

Step 3: Run Examples

# Run SEM simulation with 100 x 100 x 100 elements
bin/semproxy --ex 100 --ey 100 --ez 100 --method=sem --implem=makutu -o 2 --dt 0.001 --timemax 1.3 
# Run SEM with snapshots every 10 time iteration
bin/semproxy --ex 100 --ey 100 --ez 100 --method=sem -o 2 --dt 0.001 --timemax 2 -s --snap-interval 10 

# Run FD simulation
./src/main/fdproxy

---

CMake Options

The following options can be used to configure your build:

Option	Description
COMPILE_FD	Enable compilation of the FD proxy (default: ON)
COMPILE_SEM	Enable compilation of the SEM proxy (default: ON)
ENABLE_CUDA	Enable CUDA backend (used by Kokkos)
ENABLE_PYWRAP	Enable Python bindings via pybind11 (experimental)
USE_KOKKOS	Enable Kokkos support (serial by default, CUDA/OpenMP with flags)
USE_VECTOR	Use std::vector for data arrays (enabled by default unless Kokkos is used)

---

🐍 Python wrappers

Prerequisites

To install python requirements

pip install -r requirements.txt

Generation

The proxy must be configured with -DENABLE_PYWRAP=ON and installed via make install. Optionally, you can set -DCMAKE_INSTALL_PREFIX to where you want to deploy the application along with the python wrappers.

This will create a pyfuntides package in your install directory which contains both the solver and model pybind modules.

(.venv) [proxys]$ ls $MY_INSTALL_DIR/python/pyfuntides/
__init__.py  model.cpython-311-x86_64-linux-gnu.so  solver.cpython-311-x86_64-linux-gnu.so

This will also install kokkos in your python environment, which will point to the kokkos built by the pyfuntides app.

(.venv) [proxys]$ ls .venv/lib/python3.11/site-packages/kokkos/
__init__.py  libpykokkos.cpython-311-x86_64-linux-gnu.so  __pycache__  pytest.ini  test  utility.py

If you do not have write access on your python environment, it will install it under $MY_INSTALL_DIR/lib/python3.11/site-packages/kokkos. In that case you will have to extend your python path with this directory.

Usage

First, extend your PYTHONPATH to make the pyfuntides and adios package visible.

export PYTHONPATH=$PYTHONPATH:$MY_INSTALL_DIR/python

If needed (kokkos could not write in your python environment), also extend your PYTHONPATH to make the kokkos package visible.

export PYTHONPATH=$PYTHONPATH:$MY_INSTALL_DIR/lib/python3.11/site-packages

Then extend your LD_LIBRARY_PATH so that all libraries point to the same kokkos libraries that are installed in the lib64 folder.

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MY_INSTALL_DIR/lib64

There is no need to extend the LD_LIBRARY_PATH with the proxys libraries since the python wrappers use their RPATH to retrieve them in the lib folder.

Some examples on how to use the wrappers are available in the examples folder.

Tests & Benchmarks

All commands from this section should be executed from the repository root directory!

To install dev python packages

pip install -r requirements-dev.txt

To run basic python unit tests (default is using 6 threads)

pytest -vv -s  tests/units

To run basic python unit tests with more threads

pytest -vv -s  tests/units --threads 12

To run python benchmarks (default is using 6 threads)

pytest -vv -s tests/benchmarks/python

To run python benchmarks with more threads

pytest -vv -s tests/benchmarks/python --threads 12

To run all python benchmarks (default is using 1,2,4,8,16,32,64 threads)

python scripts/benchmarks/run_pywrap_benchmarks.py --verbose

Ploting Receivers and Snapshots

To plot the snapshots we provide a python script:

python ./scripts/adios_cartesian_snap_viz.py 201 201 201 --file snapshots.bp --slice

where 201 values should be replaced by number of nodes on x y and z. And file correpond to the snapshots.bp folder with bp5 files. The number of nodes correspond to (n_element x order) + 1.

For the receivers:

python ./scripts/adios_single_receiver_viz.py

within the folder containing the receivers.bp folder.