DOE OASIS DAV SDK project supported by the DOE PESO project.
One of the primary challenges on Frontier is handling the compiler wrappers for the CCE and AMD compilers, in particular how they wrap MPI and HIP.
Info | GCC | CCE | AMD |
---|---|---|---|
Version | v11.2 | v15.0.0 | v5.3.0 |
Cray MPICH | v8.1.23 | v8.1.23 | v8.1.23 |
ROCm | v5.3.0 | v5.3.0 | v5.3.0 |
Project | GCC | GCC + ROCm | CCE | CCE + ROCm | AMD | AMD + ROCm |
---|---|---|---|---|---|---|
ADIOS2 | ✅ | (N/A) | ✅ | (N/A) | ✅ | (N/A) |
HDF5 | ✅ | N/A | ✅ | N/A | ✅ | N/A |
PNetCDF | ✅ | N/A | ✅ | N/A | ✅ | N/A |
Ascent | ✅ | (🔎) | (🔎) | (🔎) | (🔎) | (🔎) |
DIY | (🔎) | (🔎) | (🔎) | (🔎) | (🔎) | (🔎) |
ParaView | ✅ | ✅ | (✅) | (✅) | (✅) | (✅) |
VisIt | ✅ | (N/A) | 🔎 | (N/A) | 🔎 | (N/A) |
VTK-m | ✅ | (✅) | 🔎 | (✅ ✅) | 🔎 | (✅ ✅) |
ZFP | ✅ | (N/A) | 🔎 | (N/A) | 🔎 | (N/A) |
✅ - Verified 🔎 - In Progress 🚫 - Broken N/A - Not Applicable
SYCL - SYCL extensions are implemented and tested using oneAPI.
ADIOS2 - ADIOS2 ROCm/HIP support is not expected in near term release.
UnifyFS - UnifyFS has a link error when using the Cray wrappers for the Cray and AMD toolchains.
Ascent - Using the CCE toolchain has issues locating MPI with CMake.
Ascent - Ascent spack recipe does not have support for ROCm. It is in under development in the Alpine Spack fork.
DIY - Not yet in DAV SDK.
ParaView - OSPRay support has been disabled due to build errors.
VisIt - VisIt utilizes a VTK version locked to a Python that is not compatible with the Python requirements of PyH5, the HDF5 python interface used by Cinema.
VisIt - VTK-m enabled GPU support for CUDA and ROCm is available VisIt, but is not officially tested as part of the DAV-SDK due to (HDF5 conflict) only recently being resolved.
VTK-m - Spack ROCm does not provide OpenMP correclty for VTKm HIP modules.
VTK-m - Frontier requires additional workarounds in the Spack recipe for VTKm that are specific to the Cray wrappers on Frontier. patch
ZFP - ZFP ROCm support is under development.