# Changing preemption order for a queue **URL:** \ **Category:** Users/Site Administrators\ **Created:** [August 25, 2021, 9:22am UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772 "2021-08-25T09:22:29Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![fnevgeny](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/fnevgeny/32/375_2.png) [@fnevgeny](https://community.openpbs.org/u/fnevgeny)\ **Post date:** [August 25, 2021, 9:22am UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/1 "2021-08-25T09:22:29Z")
Hi all, Is it possible to change the preemption order (or, for that matter, disable one of the methods) for a specific queue? Say, I have a dedicated “gpu” queue where the “suspend” method makes no sense, but is fine for “normal”, CPU-based jobs on other queues. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [August 25, 2021, 1:33pm UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/2 "2021-08-25T13:33:07Z")
Please check “preept\_targets” from this guide [https://www.altair.com/pdfs/pbsworks/PBSAdminGuide2021.1.pdf](https://www.altair.com/pdfs/pbsworks/PBSAdminGuide2021.1.pdf) and also section “4.3.5.8 Preventing Jobs from Being Preempted” example: ` qmgr -c "s q highpriorityqueue resources_default.preempt_targets=queue=lowpriorityq"` Jobs in highpriorityqueue or an express queue can only preempt jobs from the lowpriorityq queue. ---
**Author:** ![fnevgeny](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/fnevgeny/32/375_2.png) [@fnevgeny](https://community.openpbs.org/u/fnevgeny)\ **Post date:** [August 25, 2021, 2:07pm UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/3 "2021-08-25T14:07:31Z")
Thanks, but it doesn’t help. I don’t want to disable preemption entirely, just remove the “suspend” method for all jobs in a given queue (preemption through checkpointing and killing should remain). For other queues, suspending would remain a valid option. ---
**Author:** ![dtalcott](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/dtalcott/32/410_2.png) [@dtalcott](https://community.openpbs.org/u/dtalcott)\ **Post date:** [August 25, 2021, 4:27pm UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/4 "2021-08-25T16:27:33Z")
It might be overkill, but you could use the “multisched” feature to run a separate scheduler just for your GPU queues. This second scheduler could be configured without suspending as an option. See section 4.2 of the Admin Guide. ---
**Author:** ![fnevgeny](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/fnevgeny/32/375_2.png) [@fnevgeny](https://community.openpbs.org/u/fnevgeny)\ **Post date:** [August 25, 2021, 5:45pm UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/5 "2021-08-25T17:45:30Z")
Thanks for the suggestion, @dtalcott. It indeed sounds like overkill, but probably that is the only way. Strange, all that is missing is an “on\_preemption” hook event, but it seems there is nothing like that… Am I the only one with such a request? How do other people deal with preemption in mixed CPU/GPU environments? ---
**Author:** ![fnevgeny](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/fnevgeny/32/375_2.png) [@fnevgeny](https://community.openpbs.org/u/fnevgeny)\ **Post date:** [August 26, 2021, 3:38pm UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/6 "2021-08-26T15:38:40Z")
Would it be legitimate to have a routing queue having two destinations on _different_ partitions, managed by two different schedulers? I cannot find a definite answer in the docs. ---
**Author:** ![dtalcott](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/dtalcott/32/410_2.png) [@dtalcott](https://community.openpbs.org/u/dtalcott)\ **Post date:** [September 2, 2021, 4:21am UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/7 "2021-09-02T04:21:04Z")
You can set up a routing queue with multiple destinations on different partitions. However, normal behavior is (per Admin Guide 2.3.6): “Tries destinations in round-robin fashion, in the order listed” So, you would need your queues set up so queue A would reject jobs you want to run on queue B, and vice versa. Instead of this, I would write a queuejob hook that examines the job at qsub time and moves it to the desired queue. (Hooks Guide, 2.4.1 and examples 9-6 and 9-8) ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [September 2, 2021, 9:27am UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/8 "2021-09-02T09:27:59Z")
+1 @dtalcott and the below resource usage limit might help for the routing execution queues **queued\_jobs\_threshold** The maximum number of jobs that can be queued. At the server level, this includes all jobs in the complex. Queueing a job includes the qsub and qmove commands and the equivalent APIs. **queued\_jobs\_threshold\_res.** The maximum amount of the specified resource that can be allocated to queued jobs. At the server level, this includes all jobs in the complex. Queueing a job includes the qsub and qmove commands and the equivalent APIs. ---
**Author:** ![fnevgeny](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/fnevgeny/32/375_2.png) [@fnevgeny](https://community.openpbs.org/u/fnevgeny)\ **Post date:** [September 2, 2021, 9:43am UTC](https://community.openpbs.org/t/changing-preemption-order-for-a-queue/2772/9 "2021-09-02T09:43:42Z")
Thanks for the clarifications. The round-robin routing is fine. I have `resources_max.ngpus = 0` in the non-GPU queues and `resources_min.ngpus = 1` in the GPU one, so the jobs are always routed correctly - at least now, with the single partition.