# Configure OpenMPI error with OpenPBS

**URL:** <https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572>\
**Category:** Users/Site Administrators\
**Created:** [April 13, 2021, 6:43am UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572 "2021-04-13T06:43:55Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![tiger871019](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/tiger871019/32/405_2.png) [@tiger871019](https://community.openpbs.org/u/tiger871019)\
**Post date:** [April 13, 2021, 6:43am UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/1 "2021-04-13T06:43:55Z")

</div>

OpenMPI installation error

[root@cn01 ~]# yum install openmpi openmpi-devel  
Last metadata expiration check: 18:04:39 ago on Mon 12 Apr 2021 08:24:47 PM CST.  
Error:  
Problem: cannot install both hwloc-libs-2.2.0-1.el8.x86\_64 and hwloc-libs-1.11.9-3.el8.x86\_64

- package openmpi-4.0.5-3.el8.x86\_64 requires libhwloc.so.15()(64bit), but none of the providers can be installed
- package openpbs-execution-20.0.1-0.x86\_64 requires libhwloc.so.5()(64bit), but none of the providers can be installed
- cannot install the best candidate for the job
- problem with installed package openpbs-execution-20.0.1-0.x86\_64  
(try to add ‘–allowerasing’ to command line to replace conflicting packages or ‘–skip-broken’ to skip uninstallable packages or ‘–nobest’ to use not only best candidate packages)

Is there a solution for this?

---

<div class="post-metadata">

**Author:** ![speleolinux](https://avatars.discourse-cdn.com/v4/letter/s/779978/32.png) [@speleolinux](https://community.openpbs.org/u/speleolinux)\
**Post date:** [April 13, 2021, 7:24am UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/2 "2021-04-13T07:24:32Z")

</div>

Hi

I think you need to download MPI and build it from source and link in the PBS files. I have a script that I used when I compiled MPI for use with PBS version 20 in December 2020.  
I do not seem to be able to attach it so its copied in.  
Mike

```
#!/bin/bash

# This script builds OpenMPI from source. 
# Use this directly or copy it to an OpenMPI source directory and edit to suit.
#
# MPI downloaded from https://www.open-mpi.org openmpi-4.0.4.tar.gz on 3 August 2020.
#
# Usage: ./build_openmpi [clean] 
#
# Author: Mike lake
# Version: 2020.12.01

# Location of our OpenMPI source 
source="/home/XXX/src/openmpi4/openmpi-4.0.4"

# Location of where we wish to install the build. 
target="/shared/opt/openmpi-4.0.4"

# Logfile for the build.
log=openmpi.log

##############
# Start Script
##############

mkdir -p $target
cd $source 

cat /dev/null > $log

# A distclean takes a while so only do it if we 
# specify "clean" as an arg to this script.
if [[$# -eq 1 && $1 == 'clean']]; then
    echo "Cleaning" >> $log
    make distclean
fi

# We need to be on an execution node. This include file is only provided by the
# PBS execution package and not by the client package. 
if [! -e /opt/pbs/include/tm.h]; then
    echo "Error: /opt/pbs/include/tm.h not found. Are you on an execution node?" >> $log
    echo "Exiting." >> $log
    exit
fi
export CFLAGS="-I/opt/pbs/include"

# If we get to here we are probably on an execution node. OK. 

# The directory for the PBS libs is opt/pbs/lib/ and we wish to link in the library 
# libpbs.so (-l prepends "lib" and searches for libpbs.so or libpbs.a). 
export LD_LIBRARY_PATH=/opt/pbs/lib:$LD_LIBRARY_PATH 
export LDFLAGS="-L/opt/pbs/lib -lpbs -lpthread -lcrypto" 

# At the end of configure you can check the configure.log
echo "====== CONFIGURE ====== " >> $log

# Use this config to build for your standard MPI.
./configure --without-slurm --with-tm=/opt/pbs --enable-mpi-interface-warning --enable-shared --enable-static --enable-cxx-exceptions --prefix=$target | tee $log

echo "====== MAKE ====== " >> $log
make -j8

echo "====== INSTALL ====== " >> $log
sudo make install

cd ..

echo "The OpenMPI build has finished." 
echo "Read the $log for any errors."
```

---

<div class="post-metadata">

**Author:** ![tiger871019](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/tiger871019/32/405_2.png) [@tiger871019](https://community.openpbs.org/u/tiger871019)\
**Post date:** [April 13, 2021, 2:19pm UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/3 "2021-04-13T14:19:11Z")

</div>

Thank you for your answer I got the following error after running your script.  
Error: /opt/pbs/include/tm.h not found. Are you on an execution node?  
Exiting.

---

<div class="post-metadata">

**Author:** ![mkaro](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/mkaro/32/85_2.png) [@mkaro](https://community.openpbs.org/u/mkaro)\
**Post date:** [April 13, 2021, 3:54pm UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/4 "2021-04-13T15:54:40Z")

</div>

You need to install the openpbs-devel package to get tm.h.

---

<div class="post-metadata">

**Author:** ![speleolinux](https://avatars.discourse-cdn.com/v4/letter/s/779978/32.png) [@speleolinux](https://community.openpbs.org/u/speleolinux)\
**Post date:** [April 13, 2021, 11:33pm UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/5 "2021-04-13T23:33:49Z")

</div>

mkaro is correct:

```
$ dnf list installed | grep pbs
 pbspro-devel.x86_64 2020.1.2.20210122161823-0.el8
$ rpmquery -ql pbspro-devel.x86_64 | grep tm.h
/opt/pbs/include/tm.h

```

You can install the devel package (perhaps on just the login node?). The “make install” installs to a shared NFS drive across all the exec nodes. Sorry if I have anything wrong in my script which misleads you.

Mike

---

<div class="post-metadata">

**Author:** ![tiger871019](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/tiger871019/32/405_2.png) [@tiger871019](https://community.openpbs.org/u/tiger871019)\
**Post date:** [April 14, 2021, 5:17am UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/6 "2021-04-14T05:17:51Z")

</div>

Thank you for your answer,Is there any way I can test if the installation is complete?

---

<div class="post-metadata">

**Author:** ![speleolinux](https://avatars.discourse-cdn.com/v4/letter/s/779978/32.png) [@speleolinux](https://community.openpbs.org/u/speleolinux)\
**Post date:** [April 14, 2021, 6:40am UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/7 "2021-04-14T06:40:52Z")

</div>

Hi

```
#!/bin/bash
# Just tests MPI using python --version
#PBS -N Test
#PBS -l walltime=00:05:00 
#PBS -l select=3:mpiprocs=2:mem=1GB
#PBS -l place=scatter
    
export PATH="/shared/opt/openmpi-4.0.4/bin:$PATH"
export LD_LIBRARY_PATH="/shared/opt/openmpi-4.0.4/lib"

cd ${PBS_O_WORKDIR}
mpiexec /usr/bin/python3 --version 
cat $PBS_NODEFILE

```

When you use qsub to submit the job the output will be like this:

```
$ cat Test.o138900
Python 3.6.8
Python 3.6.8
Python 3.6.8
Python 3.6.8
Python 3.6.8
Python 3.6.8
hpcnode03
hpcnode03
hpcnode04
hpcnode04
hpcnode06
hpcnode06

```

It ran two MPI instances on on 3 nodes

---

<div class="post-metadata">

**Author:** ![tiger871019](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/tiger871019/32/405_2.png) [@tiger871019](https://community.openpbs.org/u/tiger871019)\
**Post date:** [April 16, 2021, 8:24am UTC](https://community.openpbs.org/t/configure-openmpi-error-with-openpbs/2572/8 "2021-04-16T08:24:45Z")

</div>

Thank you all for your help, I have completed the build.