# Pbs\_python external modules

**URL:** <https://community.openpbs.org/t/pbs-python-external-modules/148>\
**Category:** Developers\
**Created:** [July 13, 2016, 1:10pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148 "2016-07-13T13:10:41Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Phhere](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/phhere/32/25_2.png) [@Phhere](https://community.openpbs.org/u/Phhere)\
**Post date:** [July 13, 2016, 1:10pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/1 "2016-07-13T13:10:41Z")

</div>

Hello,  
we are using PBSPro 12 since 2012 at our univerity cluster and are really happy 🙂

We are currently trying to integrate some hooks with connection to an mysql system. But in the Administrator Guide (6.16.1) it says that only a limited number of modules are available in pbs\_python

Does this have changed in releases after PBSPro 12 ? Which version of python is currently included?

Do you have any idea how it could work else?

Thank you for your help

Greetings from Düsseldorf, Germany

---

<div class="post-metadata">

**Author:** ![ashwathraop](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/ashwathraop/32/79_2.png) [@ashwathraop](https://community.openpbs.org/u/ashwathraop)\
**Post date:** [July 13, 2016, 3:46pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/2 "2016-07-13T15:46:35Z")

</div>

Hello,

I am very glad to know that you are happy with PBSPro. Please do try the recent PBSPro version as well and I am very sure you will like the latest version even more.

You could always import modules at different location by doing something like below in the hook itself:  
import sys  
sys.path.append(“path\_to\_modules”)  
import custom\_module as module\_name

Many things changed since version 12. Currently PBSPro does not ship python with package but uses python (_should be version \>=2.6 and \<3_) installed on the system itself.

I hope this will help you 🙂

Thanks,  
Ashwath Rao

---

<div class="post-metadata">

**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\
**Post date:** [July 13, 2016, 4:18pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/3 "2016-07-13T16:18:58Z")

</div>

Hi. Version 13 of PBS Pro still has the restricted set of Python modules that can be used, and is shipped with Python 2.5.1. But after that, in the open source version, there shouldn’t be any restrictions anymore. It should be able to access to any modules that come with the Python used (\>2.6, \< 3, preferably Python 2.7) to compile PBS against.

---

<div class="post-metadata">

**Author:** ![Phhere](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/phhere/32/25_2.png) [@Phhere](https://community.openpbs.org/u/Phhere)\
**Post date:** [July 13, 2016, 5:59pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/4 "2016-07-13T17:59:59Z")

</div>

Thank you for your information!

Maybe i can split out a part of our cluster to test the installation.  
Is there any guide for an upgrade from 12 to 14 (OpenSource) ?

---

<div class="post-metadata">

**Author:** ![mkaro](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/mkaro/32/85_2.png) [@mkaro](https://community.openpbs.org/u/mkaro)\
**Post date:** [July 13, 2016, 6:28pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/5 "2016-07-13T18:28:56Z")

</div>

PBS Pro 14 is effectively the same thing as PBS Pro 13, but geared for the open source community. You may find the documentation for PBS Pro 13 (including the installation guide) here: [http://www.pbsworks.com/SupportGT.aspx?d=PBS-Professional,-Documentation](http://www.pbsworks.com/SupportGT.aspx?d=PBS-Professional,-Documentation)

Because the open source release does not contain packages such as PostgreSQL, it requires them externally. Depending on the OS you are running, there may be differences in versions of these components. Obviously, we can’t test every combination so please proceed with caution. It is possible that upgrading an existing installation could corrupt the PBS Pro database, so I encourage you to backup anything you might need if things don’t go as expected.

I would recommend that you generate RPMs so you can easily install the PBS Pro components on multiple systems. There are instructions for generating the RPMs here: [https://pbspro.atlassian.net/wiki/display/PBSPro/Building+PBS+Pro+Using+rpmbuild](https://pbspro.atlassian.net/wiki/display/PBSPro/Building+PBS+Pro+Using+rpmbuild)

Once the RPMs are installed, you may need to update /etc/pbs.conf and PBS\_HOME/mom\_priv/config prior to starting the PBS Pro services. If the services fail to start you should see instructions on what needs to be updated.

If anything goes wrong, please let us know so we can get it fixed.

---

<div class="post-metadata">

**Author:** ![Phhere](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/phhere/32/25_2.png) [@Phhere](https://community.openpbs.org/u/Phhere)\
**Post date:** [July 13, 2016, 8:30pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/6 "2016-07-13T20:30:52Z")

</div>

Thank you for your explanation.

I will have a look into the manuals and check if i can realize my ideas with it.

We want a pre execution hook which can fail a node just before starting the job on a node. But the runjob hook does not run on every vnode and the execjob\_begin hook does not have the right to change vnode state. Any idea?

---

<div class="post-metadata">

**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\
**Post date:** [July 13, 2016, 8:47pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/7 "2016-07-13T20:47:57Z")

</div>

If you use any of the mom hooks, in particular execjob\_begin hook, you’re only allowed to change the node state of those nodes managed by the local mom, where the hook is executing. The execjob\_begin hook runs on the other sister moms as well, so perhaps you can design the execjob\_begin hook to detect which host it is executing, and change the node state of the nodes the mom is managing. When walking through the list of nodes assigned to the job via pbs.event().vnode\_list, you’ll find the vnodes that are managed locally have attributes and resources\_available.\* set.

---

<div class="post-metadata">

**Author:** ![Phhere](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/phhere/32/25_2.png) [@Phhere](https://community.openpbs.org/u/Phhere)\
**Post date:** [July 13, 2016, 8:57pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/8 "2016-07-13T20:57:04Z")

</div>

I tried to use execjob\_begin together with `pbs.vnode.state = pbs.ND_OFFLINE` and event.reject() but it does not work in PBS 12

The Programmers Guide of PBS 13 says that this should be possible (6.8.9.2) with execjob\_begin in combination with fail\_action for the hook. Does anyone has tested it ?

---

<div class="post-metadata">

**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\
**Post date:** [July 13, 2016, 9:19pm UTC](https://community.openpbs.org/t/pbs-python-external-modules/148/9 "2016-07-13T21:19:21Z")

</div>

Sorry for the confusion in the documentation. The pbs.vnode in this case refers to an actual pbs vnode object obtained from:

vn = pbs.server().vnode(“vnode\_name”)  
vn.state = pbs.ND\_OFFLINE

or in a mom hook:

for v in pbs.event().vnode\_list.keys():  
[indent]pbs.event().vnode\_list[v].state = pbs.ND\_OFFLINE

In terms of fail\_action, one can set the hook’s attribute to “offline\_vnodes”:

qmgr -c “set hook \<hook\_name\> fail\_action = offline\_vnodes”

which will cause the vnodes managed by the mom executing the hook to get offlined when the hook encounters an unhandled exception or alarm timeout.
