Soft-like Requirement for Resource

Users/Site Administrators
1

I have a host-level resource called “speed” that is either fast or normal. Some users want to be able to request a “fast” host but if there are none available, get scheduled on a normal host.

Is there a way, either in qsub or the scheduler to have that resource be a “soft” requirement? Or would this be better served via placement sets?

2

soft limit can be applied to the consumable resources.

Solution1:

  • qsub wrapper script which checks the current availability and updates the qsub request and submits the job ( aware of race condition when multiple users are submitting job )

Solution 2:
Create a 3 queues:

  1. fastq – mapped to fast nodes (default_chunk.speed=fast )
  2. slowq – mapped to slow nodes (default_chunk.speed=slow)
  3. anyq – mapped to all nodes (default chunk speed=any)

node1 – resources_available.speed=fast,all
node2 – resources_available.speed=slow,all

  1. server periodic hook, which check if job in fastq has not fast nodes, it qmoves the job to anyq and the same for slowq and slow nodes
3

Hi Adarsh,
Thanks for the info. I’m leaning Solution 2, one question:

  1. anyq - mapped to all nodes (default_chunk.speed=all) is the correct syntax right? Provided you set nodeX - resources_available.speed=(fast/slow),all
1 Like
4

Yes, your understanding is correct