# Modules unavailable when loading from PBS script

**URL:** <https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893>\
**Category:** Users/Site Administrators\
**Created:** [May 28, 2024, 8:53pm UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893 "2024-05-28T20:53:24Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![ryan](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ryan](https://community.openpbs.org/u/ryan)\
**Post date:** [May 28, 2024, 8:53pm UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/1 "2024-05-28T20:53:24Z")

</div>

Hello,

We are experiencing an issue where when we try and load modules within a PBS script, we are getting “command not found” for the module command. See below.  
[randerson10@login0002 Python\_examples]$ qsub basic\_python.pbs  
40.wwadm01  
[randerson10@login0002 Python\_examples]$ cat test.e40  
/var/spool/pbs/mom\_priv/jobs/40.wwadm01.SC: line 14: module: command not found  
/var/spool/pbs/mom\_priv/jobs/40.wwadm01.SC: line 18: python: command not found

We are able to run the module command from the shell with regular users on the compute nodes and all our software shows up and loads flawlessly.  
[randerson10@node0001 ~]$ module load go  
[randerson10@node0001 ~]$ module list  
Currently Loaded Modules:

1. python/3.8.6-intel-uly7 2) go/1.19.1

Both $PATH’s for regular users and root contain the locations for our modules.

Is there any explanation for this behavior? Modules unable to load when trying to be loaded by scripts

---

<div class="post-metadata">

**Author:** ![dtalcott](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/dtalcott/32/410_2.png) [@dtalcott](https://community.openpbs.org/u/dtalcott)\
**Post date:** [May 29, 2024, 12:24am UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/2 "2024-05-29T00:24:25Z")

</div>

It appears the shell environment is not being set as expected for your job. My guess is that the shell startup files are not being run. To help diagnose this, edit your basic\_python.pbs script so that the first executable lines are

/usr/bin/ps -fu randerson10  
/usr/bin/env

Also, just to verify which module command you are expecting, on your terminal run

module --version

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [May 29, 2024, 6:36am UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/3 "2024-05-29T06:36:53Z")

</div>

+1 @dtalcott

Also you can try to launch interactive test job and check whether you can load module

Test1 : ` qsub -I`  
Test2 : `qsub -V -I`

---

<div class="post-metadata">

**Author:** ![Bert](https://avatars.discourse-cdn.com/v4/letter/b/9e8a1a/32.png) [@Bert](https://community.openpbs.org/u/Bert)\
**Post date:** [May 29, 2024, 12:47pm UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/4 "2024-05-29T12:47:54Z")

</div>

You could try to use the absolute path!?

---

<div class="post-metadata">

**Author:** ![ryan](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ryan](https://community.openpbs.org/u/ryan)\
**Post date:** [May 29, 2024, 1:35pm UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/5 "2024-05-29T13:35:12Z")

</div>

Adding /usr/bin/ps -fu randerson10 and /usr/bin/env had not affect.

The module --version output just is the following. module --avail, list, load is what is expected.  
Modules based on Lua: Version 8.7.32 2023-08-28 12:42 -05:00  
by Robert McLay mclay@tacc.utexas.edu

I am using OpenHPC with LMOD, and the **lmod.sh** withing **/etc/profile.d/** file is below. It looks alright to my eye and it has not been messed with so it is the default that comes with.

[root@login0002 init]# cat /etc/profile.d/lmod.sh

#!/bin/sh

# -_- shell-script -_-

########################################################################

# This is the system wide source file for setting up

# modules:

# 

########################################################################

# NOOP if running under known resource manager

if [-n “$SLURM\_NODELIST”] || [-n “$PBS\_NODEFILE”]; then

return

fi

export LMOD\_SETTARG\_CMD=“:”

export LMOD\_FULL\_SETTARG\_SUPPORT=no

export LMOD\_COLORIZE=no

export LMOD\_PREPEND\_BLOCK=normal

if [$EUID -eq 0]; then

export MODULEPATH=/opt/ohpc/admin/modulefiles:/opt/ohpc/pub/modulefiles

else

export MODULEPATH=/opt/ohpc/pub/modulefiles

fi

export BASH\_ENV=/opt/ohpc/admin/lmod/lmod/init/bash

# Initialize modules system

. /opt/ohpc/admin/lmod/lmod/init/bash \>/dev/null

# Load baseline OpenHPC environment

module try-add ohpc

---

<div class="post-metadata">

**Author:** ![dtalcott](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/dtalcott/32/410_2.png) [@dtalcott](https://community.openpbs.org/u/dtalcott)\
**Post date:** [May 29, 2024, 11:34pm UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/6 "2024-05-29T23:34:14Z")

</div>

I’m not familiar with ohpc and LMOD, but the following from /etc/profile.d/lmod.sh looks suspect:

> [@ryan](#):
>
> ```auto
> if [-n “$SLURM_NODELIST”] || [-n “$PBS_NODEFILE”]; then
> return
> fi
> 
> ```

Given that PBS sets PBS\_NODEFILE, this says that lmod deliberately skips setting up modules when running under PBS. I have no idea why.

As adarsh suggested, if you run an interactive job (`qsub -I`), is the module command available?

In any case, put the following near the top of your job script and see if it helps:

`source /opt/ohpc/admin/lmod/lmod/init/bash >/dev/null`

---

<div class="post-metadata">

**Author:** ![ryan](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ryan](https://community.openpbs.org/u/ryan)\
**Post date:** [May 30, 2024, 1:27pm UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/7 "2024-05-30T13:27:00Z")

</div>

So running an interactive job on the node is not working as you thought.

I am unable to load any modules because the module command is not available.

I tried putting **source /opt/ohpc/admin/lmod/lmod/init/bash \>/dev/null** but that is not fixing the problem of initializing the module environment.

The lmod.sh script I referenced above was only on the login node, but placing it on the compute nodes now allows me to run an interactive job and then the module command is available. However, when submitting a non-interactive job, my job script error output still comes back with **module command not found**.

Any ideas as to why that would be?

---

<div class="post-metadata">

**Author:** ![dtalcott](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/dtalcott/32/410_2.png) [@dtalcott](https://community.openpbs.org/u/dtalcott)\
**Post date:** [May 31, 2024, 4:15am UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/8 "2024-05-31T04:15:34Z")

</div>

Using google, I found that this is a known issue with openhpc.

[https://lists.openhpc.community/g/users/topic/18100521](https://lists.openhpc.community/g/users/topic/18100521)

My suggestion: Make a copy of /etc/profile.d/lmod.sh, but remove the three lines mentioned above:

```auto
if [-n “$SLURM_NODELIST”] || [-n “$PBS_NODEFILE”]; then
    return
fi

```

Then, at the start of your pbs script source that copy:

`source my_lmod.sh`

What is happening is that lmod is expecting PBS to forward your entire environment when you qsub the job. This is not default behavior, but you can force it by adding -V to the qsub arguments. However, because no other shell startup scripts have a similar behavior, you’ll end up with a mixture of the qsub environment and the node environment.

---

<div class="post-metadata">

**Author:** ![ryan](https://avatars.discourse-cdn.com/v4/letter/r/a8b319/32.png) [@ryan](https://community.openpbs.org/u/ryan)\
**Post date:** [June 3, 2024, 1:20pm UTC](https://community.openpbs.org/t/modules-unavailable-when-loading-from-pbs-script/3893/9 "2024-06-03T13:20:24Z")

</div>

Ahhh I see. Yes, those lines definitely looked to be part of the problem.

I commented out that conditional and sources the lmod.sh file in the job script.

I am now getting the following error when submitting a basic python job. I believe the module environment is still not getting processed correctly.

/var/spool/pbs/mom\_priv/jobs/99.wwadm01.SC: line 20: 343688 Illegal instruction (core dumped) python hello.py

My entire job script is here.

!/bin/bash  
#PBS -q default  
#PBS -N test

## serial jobs: ONLY 1 processor core is requested

#PBS -l select=1:mem=2gb:ncpus=1  
#PBS -l walltime=08:00:00  
#PBS -m abe  
#PBS -W group\_list=x-ccast-prj-saula  
#PBS -o test

source /etc/profile.d/lmod.sh

module load python/3.8.6-gcc-2pmf

cd $PBS\_O\_WORKDIR

python hello.py