License check and fetch the 3rd party license

Users/Site Administrators
1

Hi

I want to fetch the license in particular queue.
we submits the jobs it will check a licenses available and fetch the 3rd party licenses in queue.
How to configure the OpenPBS to fetch the licenses ?

2

Please follow this link and discussion:

3

Hi Adarsh,
Thanks for your reply.
We have done the license availability check.
We want to know how to fetch the licenses or hold the licenses for particular queue jobs with users?

Actually we are trying run parallel jobs with 10 licenses. Example: we submitted 10 jobs with 10 licenses, 1st job will be run with 10 license remaining jobs should be goes to queue state once 1st job completed after that 2nd job need to run with 10 licenses then same for remaining jobs will execute.

But from our side all jobs goes to running state because scripts compile time take 5 minutes then only licenses will fetch. So its not going to queue state.

Please suggest how to fix this?

4

Hi Ramesh,

For this scenario, implement server level resource ( with an assumption 10 licenses are reserved for the cluster usage and no user is checking out this license externally on to his workstation or laptop)

  • qmgr : create resource liccheck type=long, flag=q
  • qmgr : s s resources_available.liccheck=10
  • Add liccheck to the $PBS_HOME/sched_priv/sched_config resources: line
    resources: “ncpus, mem, arch, host, vnode, aoe, eoe, liccheck”
  • kill -HUP

Now while requesting the license for the job

qsub -l select=1:ncpus=10 -l your_server_dyn_res_custom_resource=10 -l licheck=10 job script

or if you are using directives in a script

#PBS -l select=1:ncpus=10
#PBS -l your_server_dyn_res_custom_resource=10
#PBS -l liccheck=10

Hope this helps

5

It is the application that is fetching the licenses, PBS Pro does not fetch or hold any licenses on behalf of the application. You might need to implement signal handlers within your script so that when the application recieves a suspend signal (man qsig), then it should release the licenses.

Correct, When the scheduler queries the license server for these jobs (via the server_dyn_scripts), the licenses are available (none of the jobs have checkedout the licenses) and hence jobs are let to run on the compute resources (one of the job gets the license and other jobs fails). PBS Pro does not know that the application takes 5 minutes to checkout the licenses, it is looking at the response of the license server with the available number of tokens.

6

Hi Adarsh,
Thank for the clarification,
We have changed our scripts, Now our scripts pick 10 license for a job.
Now we submitted multiple jobs but it has picked all 100 licenses for 10 jobs.

We are trying to execute job priority wise. Example
We have submit jobA, jobB and jobC
jobA should be run and other jobs should be in queue. Once jobA completed next jobB will be run and jobC same in q state.
JobA → JobB → JobC

Please help me how to configure the logic.

7

For example , the user are submitting these jobs (A, B, C) to a queue , say for example “workq”

#Limit the total number of running jobs for all users to 1 (this will help the licensing bit as well, for late checkout of licenses by application)
qmgr -c "set queue workq max_run = ‘[o:PBS_ALL=1]’ "

#You can use the job sort formula to sort the jobs based on desired factors
qmgr -c “set server job_sort_formula=ncpus * walltime + queue_priority”
Note: Make sure by_queue scheduler attritube is set to false in the $PBS_HOME/sched_priv/sched_config and kill -HUP , so that all jobs belong one big bucket (otherwise, it will be queue by queue basis)

Please refer the PBS Professional Adminstrator guide and check for

  • server and queue limits
  • job sort formula