PBS pick the amount of execution nodes needed

Users/Site Administrators
1

Hello,

I would like to know it there is any way to have the PBS scheduler or queue adjust the amount of compute nodes needed based on what’s free. As an example, if I want to execute a job with 96 cpu’s and I have the following compute node profiles available:

Group48 - nodes with 48 cores
Group24 - nodes with 24 cores
Group16 - nodes with 15 cores

I would like the job to run with 2 X 48, if not available then 4 X 24, if not available then 6 X 16.

Thank you

2

You can do this

  • in the queuejob or runjob hook and modifying the select statement according to the availability of the nodes ( by having a script that will query the underlying availability of nodes and helps in constructing in select statement) .
  • you can create list of profiles when such requests are made, which profile of select statement to use to satisfy the below
1 Like
3

Hi @hpcc,

There is an ongoing discussion about an enhancement related to your request (latest is at PP-507: Add support for filtering nodes as per the job request). Depending on your high-level use case (the goal you are trying to achieve), it might provide a better solution (in a future version of PBS Pro). To that end, it would be really useful if you could take a few minutes and describe your use case in more detail.

Ideally, describing what you are trying to achieve at a higher level (above the implementation level). For example, what kind of application is being run? Are you expecting it to execute faster when run on fewer nodes – is that the main goal? Or, are you trying to bin-pack the machine, using fewer nodes, with the goal of better overall utilization? When you say “if not available then …”, do you mean “if not available when the job is submitted”, or “if not available when the PBS Pro scheduler tries to start it”, or “if not available ever” or ?. Basically more background and high-level goals would help in making and refining design decisions for the future enhancement to PBS Pro.

Thanks!

1 Like
4

Thank you adarsh. That sounds like a great direction to go.

1 Like