Hi,
I’m running MPI applications across multiple nodes on a HPC cluster with InfiniBand, using SLURM and Singularity containers.
My typical SLURM command within an sbatch script is:
srun -n $SLURM_NTASKS --mpi=pmix singularity exec ...
This example uses PMIx (but it also works with PMI2) to handle non-ABI-compatible MPI variants between the host and container, relying solely on the container’s MPI and PMIx libraries for bitwise reproducibility (no host MPI bindings).
What would be the equivalent command for PBS?
Additionally, have some of you already experience with that, and if yes, how did it perform compared to running the same MPI application directly on the host (without Singularity)?
Any insights, example scripts, or configuration tips (e.g., PBS directives, PMIx setup, InfiniBand optimization) would be greatly appreciated.
Thanks