Script work does not work when invoked as hook

Users/Site Administrators
1

Hey -

Considering this very simple YAML script

Hello:
   1: a
   2: b
   3: c

I can read it successfully using the python3 code below

import yaml

queue_settings_file = "test.yaml"
reader = open(queue_settings_file, "r")
try:
    docs = yaml.load_all(reader, Loader=yaml.FullLoader)
except Exception as err:
    message = "Unable to open " + queue_settings_file + " due to " +str(err)

for doc in docs:
        print(doc)

Now I slightly adjust it to create a PBS hook:

import yaml
import pbs

e = pbs.event()
queue_settings_file = "test.yaml"
reader = open(queue_settings_file, "r")
try:
    docs = yaml.load_all(reader, Loader=yaml.FullLoader)
except Exception as err:
    message = "Unable to open " + queue_settings_file + " due to " +str(err)
    e.reject(message)

I configure it as queuejob hook and simply try to submit a job, but I’m getting the following error message:

qsub: Unable to open test.yaml due to 'module' object has no attribute 'load_all'

I have confirmed sys.executable is pointing to /usr/bin/python, so not sure what’s happening here. In addition of yaml, I also noticed my ldap module is not correctly recognized when the code is triggered by pbs hook. Am I missing something about the PBS hook environment? Do I need to sys.path.append() specific location or similar?

It seems like PBS Python is configured with Python2 environment even though my /bin/python is python3. Is there a way to force PBS Python to be Python3 ?

2

You might have to install pyyaml for Python 2.7x and include it in the site packages (/opt/pbs/python/lib/python2.7 ) and have to give it a try.

3

Yup just noticed even though I use py3 my PBS was compiled with py27 … I tried to compile PBSPRO (including v19.x) but it does not seems to support py3 yet.

Do we know when PBSPro will support py3 knowing py2 is EOL ?

4

Upcoming releases in 2020 would be Python 3 complaint

5

Awesome, I will keep an eye on https://github.com/PBSPro/pbspro/releases then !

Thanks

1 Like