PP-239 and PP-659: Decorator to skip PTL tests on cpuset mom; Specifying default test case time out value while running PTL tests

Developers
1

Hi @developers,

Please review a new design document created for two aspects of PTL framework:

Thanks,
Sarita

2

@saritakh

We have been seeing that there are bunch of custom decorators related to skipping the test on a specific platform or system.
Instead of adding a bunch of them can’t we just have a generic decorator? something like “SkipOn” and we pass on values like “Cray”, “cpuset”, “non-cray” etc. to it.

This will save us from creating a bunch of decorators and essentially all these actually do the same thing, which is skip the test when certain condition is met.

My proposal is to make it generic rather than adding plenty of decorators.

1 Like
3

Is a decorator the right approach? Is it possible for the operations that operate on or depend on the presence of a CPUset mom to autodetect the fact that the deployment is running a standard mom and return a useful error that the test can use to skip and exit gracefully?

4

I think it is best that the test writer mentions where he/she would want to skip the test.

It is possible that a test uses generic PBS interfaces and then does something (platform dependent) outside the scope of PBS to verify whether or not PBS behaved in the correct fashion. In these cases It is only the test writer who knows where to run this test.

5

Hi Sarita, PFB some comments:

"Tests that insert new vnodes
Tests that alter node resources (like ncpus, memory etc.)"
I don’t know a lot about the restrictions on cpuset moms, so could you please clarify the following:

  • Is it just the tests which insert new vnodes? What about the ones which delete vnodes? should they be skipped too?
  • should we skip all test cases which alter ANY resource on a node? The ones that you mentioned are consumable resources, is it bad to alter even non-consumables on cpuset nodes?

“A new option added to pbs_benchpress command to increase default test case time out value”: I’m guessing that the original motivation for adding this option was to be able to increase the default test case time out, but one can set this to a really small value as well right? So, I’d change ‘increase’ to just ‘change’ or ‘set’.

6

@arungrover @iestockdale
We plan to enhance the test skip with a newer approach in future. So for this feature the current approach can be taken forward. We will revisit the same when we take up the new approach. A ticket PP-695 has been raised for the new approach:

PP-695: Enhanced approach to run or skip PTL tests based on various reasons

7

@agrawalravi90
Regarding deleting vnodes when cpuset mom is run:

  • Yes, the tests that delete vnode also need to be skipped; although I think its very rare that a test case deletes vnodes without creating them
    Regarding the tests that edit vnode resources or their values:
  • PBS automatically examines the topology of the machine, and creates vnodes to represent subsets of the machine while running cpuset mom. Hence, it is not suggested to alter them. But custom resources can be edited on the vnodes.
    Regarding the update of “increase” to “change”/ “set”
  • Agreed, I have updated the same in design document
8

@saritakh it works for me

9

@saritakh Design looks good to me.

10

@developers I have updated the design page with the specification that “default-testcase-timeout” is a parameter to option ‘-p’; which was missing previously. Please review and sign off or let me know of any comments.