Help: example PTL test

Users/Site Administrators
1

I’m working on a bug report, but would also like to create a PTL test that shows the bug. However, I don’t want to install the test in /opt/ptl/tests until I have debugged it. I’ve read through the docs under https://openpbs.atlassian.net/wiki/spaces/DG/pages/518029315/Testing+Your+Code and https://openpbs.atlassian.net/wiki/spaces/PD/pages/1319829507/Enhancement+of+pbs+benchpress+to+run+test+case+from+non-test+directory, but am not having much luck.

Could someone show me the steps to use pbs_benchpress to run a new test (assuming I have PTL running fine for the existing tests)?

For a concrete example, make a local copy of the test at /opt/ptl/tests/functional/pbs_dup_acc_log_for_resv.py, change it to have a reserve_end 61 seconds in the future, and test the result. (This is not the bug, just a simple test.)

This is what I get:

$ pwd
/home/dtalcott/work/ptl_tests
$ pbs_benchpress -f ./pbs_dup_acc_log_for_resv.py -L -l DEBUG
2021-09-08 11:50:56,177 DEBUG    included_tests:{'__only__ts__': ['PBSTestSuite'], '__only__tc__': []}
2021-09-08 11:50:56,178 DEBUG    included_tests(copy):{'__only__ts__': ['PBSTestSuite'], '__only__tc__': []}
2021-09-08 11:50:56,178 DEBUG    excluded_tests:{'__only__ts__': [], '__only__tc__': []}
2021-09-08 11:50:56,184 DEBUG    check_unknown called
2021-09-08 11:50:56,184 ERROR    unknown testsuite(s): PBSTestSuite

Nothing I have tried gets past the “unknown testsuite(s)” error. I can use -t xyz to change the suite it complains about, but that’s it. I haven’t found a value for xyz that it likes.

Thanks.

2

Looks as if the Confluence pages need an update. When you get that info, can you give it to me? Thanks.

3

If you are able to run existing tests in the same directory, then I think there might be a Python bug in your new test file. On the other hand, if your new test file lives in a different directory than the existing tests, then try pointing PYTHONPATH to where you installed the ptl module. e.g - PYTHONPATH=/opt/ptl/lib/python3.6/site-packages pbs_benchpress -f pbs_dup_acc_log_for_resv.py -L -l DEBUG

4

I don’t think that’s it:

> ls -l
total 8
-rw-r--r-- 1 501 games 2405 Sep  8 15:22 pbs_dup_acc_log_for_resv.py
-rw-r--r-- 1 501 games 2519 Sep  8 10:50 pbs_grunt.py
> echo $PYTHONPATH
:/opt/ptl/lib/python3.6/site-packages:/opt/pbs/lib/python/altair
> pbs_benchpress -f ./pbs_dup_acc_log_for_resv.py -L
2021-09-08 15:18:49,141 ERROR    unknown testsuite(s): PBSTestSuite

And, I don’t think there is a problem with the test itself. The file pbs_dup_acc_log_for_resv.py is a straight copy of /opt/ptl/tests/functional/pbs_dup_acc_log_for_resv.py.

A clue: If I introduce a python error in my file, but tell pbs_benchpress to run one of the standard tests (-t TestDupAccLogForResv.test_accounting_logs), then it runs the standard test, then issues a python error msg.

> pbs_benchpress -t TestDupAccLogForResv.test_accounting_logs
2021-09-08 16:03:20,930 INFO     input command: pbs_benchpress -t TestDupAccLogForResv.test_accounting_logs
2021-09-08 16:03:20,931 INFO     param: None
2021-09-08 16:03:20,931 INFO     ptl version: 20.0.1_lessNAS
2021-09-08 16:03:20,934 INFO     platform: Linux server2.local 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64
2021-09-08 16:03:20,934 INFO     python version: 3.6.8
2021-09-08 16:03:20,934 INFO     user: dtalcott
2021-09-08 16:03:20,934 INFO     --------------------------------------------------------------------------------
2021-09-08 16:03:20,934 INFO     Cleaning up temporary files

2021-09-08 16:03:21,343 INFO     Cleaning up /tmp dir
2021-09-08 16:03:21,343 INFO     Cleaning up /var/tmp dir
2021-09-08 16:03:22,693 INFO     ======================================================================
2021-09-08 16:03:22,694 INFO     suite name: Failure
2021-09-08 16:03:22,694 INFO     suite docstring: 
Unloadable or unexecutable test.

    A Failure case is placed in a test suite to indicate the presence of a
    test that could not be loaded or executed. A common example is a test
    module that fails to import.
    
[ ... ]
2021-09-08 16:01:48,194 INFO     =======================================
2021-09-08 16:01:48,194 INFO     Completed TestDupAccLogForResv tearDown
2021-09-08 16:01:48,195 INFO     =======================================
2021-09-08 16:01:48,199 INFO     ok

2021-09-08 16:01:48,202 INFO     
2021-09-08 16:01:48,202 INFO     ======================================================================
2021-09-08 16:01:48,202 INFO     ERROR: Failure: SyntaxError (invalid syntax (pbs_grunt.py, line 58))

2021-09-08 16:01:48,202 INFO     ___m_oo_m___

So, pbs_benchpress sees my test, but I don’t know how to get it to run it. I think my problem is that I cannot come up the right value for the -t option to pbs_benchpress.

5

hmm ok … I think the problem might be that you are running the test out of a directory that doesn’t have the functional/__init__.py, which is where we define TestFunctional, which inherits from PBSTestSuite. All the tests inside functional/ inherit TestFunctional, including TestDupAccLogForResv. What I usually do is that I copy over the pbs source tree, then go inside openpbs/test/tests/functional/ and create my new test file there. If that doesn’t work for you, then you can try adding “from ptl.utils.pbs_testsuite import *” to your test file, that should help it find PBSTestSuite.

6

I finally got it to work. Here is my dummy test, called “pbs_hello.py”

# coding: utf-8

from tests.functional import *

class TestExample(TestFunctional):

    """
    This test suite is just an example
    """

    def test_hello(self):
        """
        Produce a hello world message
        """
        our_queue = 'workq'
        our_text = 'Hello world'
        a = {'comment': our_text}
        self.server.manager(MGR_CMD_SET, QUEUE, a, id=our_queue)
        self.server.expect(QUEUE, a, id=our_queue)

Here are the steps to run it, assuming you can already run pbs_benchpress with the installed tests.

mkdir my_ptl_tests
cd my_ptl_tests/
vi pbs_hello.py

mkdir tests tests/functional
ln -s /opt/ptl/tests/functional/__init__.py  tests/functional/
ln -s ../../pbs_hello.py tests/functional/ # See note
touch __init__.py

pbs_benchpress -t TestExample.test_hello

Note: This symlink makes your test appear to be in the tests/functional hierarchy. You could put it there directly, but I prefer to work with it at the my_ptl_tests level.

It would be nice if someone could verify that it works for them also.

FWIW, My original test case now works and crashes the server, as expected. :slight_smile: