Lumerical mpi syntax mismatch with pbs mpi

Need your inputs:

pasting mpi lines from below link

General MPI Syntax

mpiexec [mpi_options] solver [solver_options]

MPI Options


  • hostfile : text file with 1 hostname/IP per line, with corresponding number of processes separated by a comma ':'

we observed:

pbs hostfile is as below


it will not work as lumerical mpi syntax

/home/apps/lumerical/v211/mpich2/nemesis/bin/mpiexec -hosts $PBS_NODEFILE /home/apps/lumerical/v211/bin/fdtd-engine-mpich2nem -t 1 -logall $PBS_O_WORKDIR/test.fsp

Can something be done here ?

PBS script:

#PBS -N testmpi
#PBS -l select=2:ncpus=32:mpiprocs=32:mem=64gb
#PBS -q test
/home/apps/lumerical/v211/mpich2/nemesis/bin/mpiexec hostname
/home/apps/lumerical/v211/mpich2/nemesis/bin/mpiexec -hosts $PBS_NODEFILE /home/apps/lumerical/v211/bin/fdtd-engine-mpich2nem -t 1 -logall $PBS_O_WORKDIR/test.fsp

Which MPI flavor are you using (MPICH, OpenMPI, etc.)? Was it compiled with task manager (TM) support for PBS/Torque? If not, it’s likely attempting to use ssh to spawn tasks on the nodes in $PBS_NODEFILE. Depending on the configuration of your nodes, this will likely fail.

You would have to read the $PBS_NODEFILE and then write into new file to your required format & use that file or create a script that produces output in the required format by the MPI. As @mkaro mentioned, if you have compiled the flavour of MPI with PBS task manager ™ , then the $PBS_NODEFILE would be automatically picked up by PBS without requiring to specify the -hosts option.

cat $PBS_NODEFILE | sort | uniq -c | awk ‘{print ($2":"$1)};’| xargs | sed -e ‘s/ /,/g’

@mkaro @adarsh its helpful pointers. thank you.