PTL testlib to alter a job

Developers
1

I would like to tweak some things on a job outside of a hook so I thought I would
try using the ptl_testlib python package. However its crashing when the underlying library is calling qalter:

ptl_testlib.Server(pbs_server_name).alterjob(jobid=jid, attrib=new_att)

gives:

PbsAlterError Traceback (most recent call last)
<ipython-input-61-48f975436349> in <module>()
----> 1 ptl_testlib.Server(pbs_server_name).alterjob(jobid=jid, attrib=new_att)

/opt/pbs/python/lib/python2.7/site-packages/ptl/lib/pbs_testlib.pyc in alterjob(self, jobid, attrib, extend, runas, logerr)
6940 if rc != 0:
6941 raise PbsAlterError(rc=rc, rv=False, msg=self.geterrmsg(),
-> 6942 post=self._disconnect, conn=c)
6943
6944 if c:

PbsAlterError: rc=2, rv=False, msg=["/opt/pbs/bin/qalter: invalid option – ‘v’", ‘usage: qalter [-a date_time] [-A account_string] [-c interval] [-e path]’, ‘\t[-h hold_list] [-j y|n] [-k keep] [-J X-Y[:Z]] [-l resource_list]’, ‘\t[-m mail_options] [-M user_list] [-N jobname] [-o path] [-p priority]’, ‘\t[-R o|e|oe] [-r y|n] [-S path] [-u user_list] [-W dependency_list]’, ‘\t[-P project_name] job_identifier…’, ’ qalter --version’]

This is using the binary rpms of 18.2 and the ptl tree from github installed as normal if that affects things any.
The qstat from github doesnt have a ‘v’ option either so at least that matches.

thoughts/comments/concerns/etc

or better ways to tweaking something on a job outside of a hook.
In this case I was just adding a variable onto the Variable_List attribute.
But I could create a new job resource and tweak that as well for what I need.

thanks

steve

2

Hi Steve, can you please post the complete log? PTL logs the exact command that it runs, so you should have seen the qalter command as PTL ran it in the logs. That will give us a better idea about what it’s doing and where the ‘v’ is coming from.

3

Logged where? There isnt anything in the server_logs files.
thanks

steve

4

sorry, can you add the following to the setup section of your script and run your script again:

from ptl.utils.pbs_cliutils import CliUtils
fmt = ‘%(asctime)-15s %(levelname)-8s %(message)s’
log_level = “INFOCLI2”
level_int = CliUtils.get_logging_level(log_level)
stream_hdlr = logging.StreamHandler()
stream_hdlr.setLevel(level_int)
stream_hdlr.setFormatter(logging.Formatter(fmt))
ptl_logger = logging.getLogger(‘ptl’)
ptl_logger.addHandler(stream_hdlr)
ptl_logger.setLevel(level_int)

it will make PTL log more information than just the exception you saw.

5

output as follows:

2018-10-11 07:40:15,884 INFOCLI hostname: /opt/pbs/bin/qstat -Bf pbs_server_name
2018-10-11 07:40:15,895 INFO server pbs_server_name: version 18.1.0
2018-10-11 07:40:15,895 INFO alter on pbs_server_name: jobid {‘Variable_List’: ‘stuff’}
2018-10-11 07:40:15,895 INFOCLI hostname: /opt/pbs/bin/qalter -v stuff jobid
2018-10-11 07:40:15,898 ERROR err: ["/opt/pbs/bin/qalter: invalid option – ‘v’", ‘usage: qalter [-a date_time] [-A account_string] [-c interval] [-e path]’, ‘\t[-h hold_list] [-j y|n] [-k keep] [-J X-Y[:Z]] [-l resource_list]’, ‘\t[-m mail_options] [-M user_list] [-N jobname] [-o path] [-p priority]’, ‘\t[-R o|e|oe] [-r y|n] [-S path] [-u user_list] [-W dependency_list]’, ‘\t[-P project_name] job_identifier…’, ’ qalter --version’]
Traceback (most recent call last):
File “./test4.py”, line 18, in
ptl_testlib.Server(pbs_server_name).alterjob(jobid=jid, attrib=new_att)
File “/opt/pbs/python/lib/python2.7/site-packages/ptl/lib/pbs_testlib.py”, line 6942, in alterjob
post=self._disconnect, conn=c)
ptl.lib.pbs_testlib.PbsAlterError: rc=2, rv=False, msg=["/opt/pbs/bin/qalter: invalid option – ‘v’", ‘usage: qalter [-a date_time] [-A account_string] [-c interval] [-e path]’, ‘\t[-h hold_list] [-j y|n] [-k keep] [-J X-Y[:Z]] [-l resource_list]’, ‘\t[-m mail_options] [-M user_list] [-N jobname] [-o path] [-p priority]’, ‘\t[-R o|e|oe] [-r y|n] [-S path] [-u user_list] [-W dependency_list]’, ‘\t[-P project_name] job_identifier…’, ’ qalter --version’]

prompt: qalter --version
pbs_version = 18.2.1.20180622093321

Note that the qalter version is different then the version ptl_testlib returns in the logging output.
not sure if this will matter but thought I would mention it.

thanks

steve

6

Thanks for providing more details. Unfortunately, it sounds like you’ve come across a bug in PTL. PTL needs to use the -W option while modifying Variable_List, it’s trying to instead use qsub’s -v option for Variable_List. I was able to successfully modify Resource_List.ncpus, but Variable_List error’d out for me in the same way.

7

At least Im not going crazy or writing bad testlib code…
btw Im happy to do this the shell way with actual qalter commands, but everything Ive
seen regarding the -W option to qalter deals with job dependencies. If there is a way to tweak
job info with it Im all for it. How however I dont know.

thanks
steve

8

I simply did the following to modify Variable_List via qalter:
qalter -W Variable_List=blah 31.pbspro

[ravi@pbspro ~]$ qstat -f | grep Variable_List
Variable_List = blah