# Mom start has python issues

**URL:** <https://community.openpbs.org/t/mom-start-has-python-issues/3464>\
**Category:** Users/Site Administrators\
**Created:** [March 14, 2023, 10:11pm UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464 "2023-03-14T22:11:43Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![steveheistand](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@steveheistand](https://community.openpbs.org/u/steveheistand)\
**Post date:** [March 14, 2023, 10:11pm UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/1 "2023-03-14T22:11:43Z")

</div>

anyone know why at job start time I might get python issues like:

03/14/2023 15:01:19;0001;pbs\_python;Svr;pbs\_python;PBS server internal error (15011) in pbs\_python\_load\_python\_types, \<class ‘ImportError’\>  
03/14/2023 15:01:19;0001;pbs\_python;Svr;pbs\_python;PBS server internal error (15011) in pbs\_python\_load\_python\_types, /opt/python/lib/python3.7/lib-dynload/math.cpython-37m-x86\_64-linux-gnu.so: undefined symbol: PyFloat\_Type  
03/14/2023 15:01:19;0001;pbs\_python;Svr;pbs\_python;pbs\_python\_ext\_start\_interpreter, could not load python types into the interpreter  
03/14/2023 15:01:19;0001;pbs\_mom;Svr;pbs\_mom;run\_hook, execv of /opt/pbs/bin/pbs\_python resulted in nonzero exit status=1

thanks  
s

---

<div class="post-metadata">

**Author:** ![steveheistand](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@steveheistand](https://community.openpbs.org/u/steveheistand)\
**Post date:** [March 14, 2023, 10:12pm UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/2 "2023-03-14T22:12:17Z")

</div>

oh and version:  
/opt/pbs/bin/pbs\_python --version  
pbs\_version = 20.0.1

---

<div class="post-metadata">

**Author:** ![dtalcott](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/dtalcott/32/410_2.png) [@dtalcott](https://community.openpbs.org/u/dtalcott)\
**Post date:** [March 16, 2023, 4:18am UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/3 "2023-03-16T04:18:41Z")

</div>

This looks like pbs\_python was built against a different version of python than it is finding at runtime. What do you get if you run the following on a MoM?

```auto
ldd /opt/pbs/bin/pbs_python

```

Do you get something different if you unset PYTHONPATH first?

Did you build PBS yourself? There is a StackOverflow article that might apply:

> <https://stackoverflow.com/questions/67891197/ctypes-cpython-39-x86-64-linux-gnu-so-undefined-symbol-pyfloat-type-in-embedd>

---

<div class="post-metadata">

**Author:** ![steveheistand](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@steveheistand](https://community.openpbs.org/u/steveheistand)\
**Post date:** [March 16, 2023, 2:15pm UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/4 "2023-03-16T14:15:11Z")

</div>

this is my initial attempts at building openpbs from source on a rocky 8 machine.  
it comes with python 3.6 installed I also dnf install python3.9. but as openpbs v20 only likes up to python3.7 I do build the latest python3.7 from source and point at it during config time when building openpbs.  
how all that combines when the mom actually starts is beyond me and obviously not quite working.

Im currently trying to switch to openpbs v22 as it likes newer python versions. which requires the PBS server to also be at v22 so its a bit more involved.

there are nice binary rpms for rocky 8 and openpbs which is generally ok for the mom side. On the server I have to build from source as I have to tweak the openpbs code so cant use rpms. at which point figure I should also build from source on the execution hosts so things get complicated…

I dont currently have an openpbs v20 machine built to see what the ldd /opt/pbs/bin/pbs\_python returns but will dump that in the issue when I get a chance.

thanks dale, (I thought you were supposed to be retired…)

---

<div class="post-metadata">

**Author:** ![steveheistand](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@steveheistand](https://community.openpbs.org/u/steveheistand)\
**Post date:** [March 16, 2023, 5:20pm UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/5 "2023-03-16T17:20:44Z")

</div>

ldd /opt/pbs/bin/pbs\_python  
linux-vdso.so.1 (0x00007ffdef28e000)  
libpthread.so.0 =\> /lib64/libpthread.so.0 (0x00007f6dbc4e2000)  
libutil.so.1 =\> /lib64/libutil.so.1 (0x00007f6dbc2de000)  
libm.so.6 =\> /lib64/libm.so.6 (0x00007f6dbbf5c000)  
libdl.so.2 =\> /lib64/libdl.so.2 (0x00007f6dbbd58000)  
libcrypt.so.1 =\> /lib64/libcrypt.so.1 (0x00007f6dbbb2f000)  
libc.so.6 =\> /lib64/libc.so.6 (0x00007f6dbb769000)  
/lib64/ld-linux-x86-64.so.2 (0x00007f6dbc702000)

unsetting PYTHONPATH did not change anything.

---

<div class="post-metadata">

**Author:** ![dtalcott](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/dtalcott/32/410_2.png) [@dtalcott](https://community.openpbs.org/u/dtalcott)\
**Post date:** [March 17, 2023, 11:42pm UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/6 "2023-03-17T23:42:29Z")

</div>

Forgive my ignorance, but why didn’t you use the python 3.6 that comes installed by default?

What does the following yield:

```auto
# nm /opt/pbs/bin/pbs_python | grep PyFloat_Type
0000000000699400 B PyFloat_Type

```

If that article I linked to explains what is going on, I would expect a different result on your system.

I’m just flailing around here. Perhaps someone with more knowledge could chime in?

---

<div class="post-metadata">

**Author:** ![steveheistand](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@steveheistand](https://community.openpbs.org/u/steveheistand)\
**Post date:** [March 18, 2023, 12:01am UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/7 "2023-03-18T00:01:52Z")

</div>

mostly I wanted a newer version of python (in addition to the required 3.6) on the machine.  
I have rolled back to not adding any newer python until after PBS is built which is helping.  
now for some strange reason a qsub hook fails with a module not found error for a python package  
that isnt imported in the hook but is installed even if it did.  
things are never simple here…  
as all this is buried in automation scripts testing something post tweak takes an hour to build up the  
environment so testing is slow…

thanks  
s

---

<div class="post-metadata">

**Author:** ![steveheistand](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@steveheistand](https://community.openpbs.org/u/steveheistand)\
**Post date:** [March 21, 2023, 7:30pm UTC](https://community.openpbs.org/t/mom-start-has-python-issues/3464/8 "2023-03-21T19:30:55Z")

</div>

followup here in case others have issues:  
so Rocky Linux 8 is stuck at python3.6. some of the packages installed to support PBS will mess with other packages depending on the order in which things are installed. I suspect as those packages have version number requirements on packages they require.  
When hooks stop working you will want to make sure you re-install any python package that hook  
(and internal PBS python usage) uses.  
And if you are in AWS reinstall the awscli.
