Help: example PTL test

I’m working on a bug report, but would also like to create a PTL test that shows the bug. However, I don’t want to install the test in /opt/ptl/tests until I have debugged it. I’ve read through the docs under and, but am not having much luck.

Could someone show me the steps to use pbs_benchpress to run a new test (assuming I have PTL running fine for the existing tests)?

For a concrete example, make a local copy of the test at /opt/ptl/tests/functional/, change it to have a reserve_end 61 seconds in the future, and test the result. (This is not the bug, just a simple test.)

This is what I get:

$ pwd
$ pbs_benchpress -f ./ -L -l DEBUG
2021-09-08 11:50:56,177 DEBUG    included_tests:{'__only__ts__': ['PBSTestSuite'], '__only__tc__': []}
2021-09-08 11:50:56,178 DEBUG    included_tests(copy):{'__only__ts__': ['PBSTestSuite'], '__only__tc__': []}
2021-09-08 11:50:56,178 DEBUG    excluded_tests:{'__only__ts__': [], '__only__tc__': []}
2021-09-08 11:50:56,184 DEBUG    check_unknown called
2021-09-08 11:50:56,184 ERROR    unknown testsuite(s): PBSTestSuite

Nothing I have tried gets past the “unknown testsuite(s)” error. I can use -t xyz to change the suite it complains about, but that’s it. I haven’t found a value for xyz that it likes.


Looks as if the Confluence pages need an update. When you get that info, can you give it to me? Thanks.

If you are able to run existing tests in the same directory, then I think there might be a Python bug in your new test file. On the other hand, if your new test file lives in a different directory than the existing tests, then try pointing PYTHONPATH to where you installed the ptl module. e.g - PYTHONPATH=/opt/ptl/lib/python3.6/site-packages pbs_benchpress -f -L -l DEBUG

I don’t think that’s it:

> ls -l
total 8
-rw-r--r-- 1 501 games 2405 Sep  8 15:22
-rw-r--r-- 1 501 games 2519 Sep  8 10:50
> pbs_benchpress -f ./ -L
2021-09-08 15:18:49,141 ERROR    unknown testsuite(s): PBSTestSuite

And, I don’t think there is a problem with the test itself. The file is a straight copy of /opt/ptl/tests/functional/

A clue: If I introduce a python error in my file, but tell pbs_benchpress to run one of the standard tests (-t TestDupAccLogForResv.test_accounting_logs), then it runs the standard test, then issues a python error msg.

> pbs_benchpress -t TestDupAccLogForResv.test_accounting_logs
2021-09-08 16:03:20,930 INFO     input command: pbs_benchpress -t TestDupAccLogForResv.test_accounting_logs
2021-09-08 16:03:20,931 INFO     param: None
2021-09-08 16:03:20,931 INFO     ptl version: 20.0.1_lessNAS
2021-09-08 16:03:20,934 INFO     platform: Linux server2.local 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64
2021-09-08 16:03:20,934 INFO     python version: 3.6.8
2021-09-08 16:03:20,934 INFO     user: dtalcott
2021-09-08 16:03:20,934 INFO     --------------------------------------------------------------------------------
2021-09-08 16:03:20,934 INFO     Cleaning up temporary files

2021-09-08 16:03:21,343 INFO     Cleaning up /tmp dir
2021-09-08 16:03:21,343 INFO     Cleaning up /var/tmp dir
2021-09-08 16:03:22,693 INFO     ======================================================================
2021-09-08 16:03:22,694 INFO     suite name: Failure
2021-09-08 16:03:22,694 INFO     suite docstring: 
Unloadable or unexecutable test.

    A Failure case is placed in a test suite to indicate the presence of a
    test that could not be loaded or executed. A common example is a test
    module that fails to import.
[ ... ]
2021-09-08 16:01:48,194 INFO     =======================================
2021-09-08 16:01:48,194 INFO     Completed TestDupAccLogForResv tearDown
2021-09-08 16:01:48,195 INFO     =======================================
2021-09-08 16:01:48,199 INFO     ok

2021-09-08 16:01:48,202 INFO     
2021-09-08 16:01:48,202 INFO     ======================================================================
2021-09-08 16:01:48,202 INFO     ERROR: Failure: SyntaxError (invalid syntax (, line 58))

2021-09-08 16:01:48,202 INFO     ___m_oo_m___

So, pbs_benchpress sees my test, but I don’t know how to get it to run it. I think my problem is that I cannot come up the right value for the -t option to pbs_benchpress.

hmm ok … I think the problem might be that you are running the test out of a directory that doesn’t have the functional/, which is where we define TestFunctional, which inherits from PBSTestSuite. All the tests inside functional/ inherit TestFunctional, including TestDupAccLogForResv. What I usually do is that I copy over the pbs source tree, then go inside openpbs/test/tests/functional/ and create my new test file there. If that doesn’t work for you, then you can try adding “from ptl.utils.pbs_testsuite import *” to your test file, that should help it find PBSTestSuite.

I finally got it to work. Here is my dummy test, called “”

# coding: utf-8

from tests.functional import *

class TestExample(TestFunctional):

    This test suite is just an example

    def test_hello(self):
        Produce a hello world message
        our_queue = 'workq'
        our_text = 'Hello world'
        a = {'comment': our_text}
        self.server.manager(MGR_CMD_SET, QUEUE, a, id=our_queue)
        self.server.expect(QUEUE, a, id=our_queue)

Here are the steps to run it, assuming you can already run pbs_benchpress with the installed tests.

mkdir my_ptl_tests
cd my_ptl_tests/

mkdir tests tests/functional
ln -s /opt/ptl/tests/functional/  tests/functional/
ln -s ../../ tests/functional/ # See note

pbs_benchpress -t TestExample.test_hello

Note: This symlink makes your test appear to be in the tests/functional hierarchy. You could put it there directly, but I prefer to work with it at the my_ptl_tests level.

It would be nice if someone could verify that it works for them also.

FWIW, My original test case now works and crashes the server, as expected. :slight_smile: