# Advanced GPU Scheduling **URL:** \ **Category:** Developers\ **Created:** [July 2, 2025, 7:13am UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156 "2025-07-02T07:13:47Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![wakaka](https://avatars.discourse-cdn.com/v4/letter/w/d07c76/32.png) [@wakaka](https://community.openpbs.org/u/wakaka)\ **Post date:** [July 2, 2025, 7:13am UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/1 "2025-07-02T07:13:47Z")
`Advanced scheduling requires defining a child vnode for each GPU。` I have two nodes, each with 18 CPUs and 4 GPUs. Is it necessary to create vnodes to use the advanced GPU scheduling resource call limitations? I just want to treat the node as a whole without subdividing it. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [July 2, 2025, 4:28pm UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/2 "2025-07-02T16:28:17Z")
1. There is no need so create vnodes to schedule jobs on to a host with multiple GPU cards The user can submit jobs by using the CUDA\_VISIBLE\_DEVICE varailable and use specific GPUs. 2. If you would like GPU device isolation then you would need to use cgroups hook and it will take care of mapping the correct CUDA\_VISIBLE\_DEVICE and/or UID of the gpu card. ---
**Author:** ![wakaka](https://avatars.discourse-cdn.com/v4/letter/w/d07c76/32.png) [@wakaka](https://community.openpbs.org/u/wakaka)\ **Post date:** [July 10, 2025, 9:14am UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/3 "2025-07-10T09:14:45Z")
1、Why does pbs\_cgroups automatically create vnodes for me when I use it, and also reduce my available memory and CPU for this node? Is vnode necessary when using cgroups? 2、When I use `-l select=1:ngpus=1`, it will use 1 cpu at the same time. How can I only use gpu without cpu? ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [July 10, 2025, 5:42pm UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/4 "2025-07-10T17:42:04Z")
Please check the documentation on **vnode\_per\_numa\_node** and **allow\_zero\_cpus** These settings would help you achieve your requirement. ---
**Author:** ![wakaka](https://avatars.discourse-cdn.com/v4/letter/w/d07c76/32.png) [@wakaka](https://community.openpbs.org/u/wakaka)\ **Post date:** [July 11, 2025, 9:57am UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/5 "2025-07-11T09:57:26Z")
Thanks for your reply, Mr.adarsh. I set the **vnode\_per\_numa\_node:false** , it doesn’t create vnode automatically, but it also reduced my memory and CPU. I originally had 4gb of memory and 4 CPUs. After enabling pbs\_cgroups, the available memory of the node was only 3gb and the available CPU was only 2. It’s a problem. How can I solve it? ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [July 11, 2025, 7:14pm UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/6 "2025-07-11T19:14:59Z")
Thank you @wakaka Please share your pbs\_cgroups.json file. - did you remove the vnodes and natural node after setting vnode\_per\_numa\_node: to false - and added the same node again, it would have created only one natural node - Also the actual mem and cpu available on the node - pbsnodes -av output ---
**Author:** ![wakaka](https://avatars.discourse-cdn.com/v4/letter/w/d07c76/32.png) [@wakaka](https://community.openpbs.org/u/wakaka)\ **Post date:** [July 14, 2025, 1:13am UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/7 "2025-07-14T01:13:36Z")
This is my pbs\_cgroups.json： ![b1189e24-c596-4228-bb42-07ef0c023d7b](https://canada1.discourse-cdn.com/flex030/uploads/pbspro/original/1X/76f3b0d213aa3eceeadb61af42719da4e7892a58.jpeg) > [@adarsh](#): > > did you remove the vnodes and natural node after setting vnode\_per\_numa\_node: to false Yeah, I have tried it, remove first and add again, but it does’t work. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [July 14, 2025, 7:53pm UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/8 "2025-07-14T19:53:39Z")
> [@wakaka](#): > > I originally had 4gb of memory and 4 CPUs. After enabling pbs\_cgroups, the available memory of the node was only 3gb and the available CPU was only 2. It’s a problem. How can I solve it? Thank you for sharing your cgroup configuration - memory subsystem set to false - not sure how this can happen reserved\_amount: 1GB: This reserved memory decreases the resources\_available.mem that MoM advertises to the server as being available for each vnode, and also reduces the amount of memory the cgroups hook will assign to jobs Also, not sure how the ncpus got decreased Probably, better to disable the cgroups hook, delete the node and add the node. Make sure this is displaying the correct mem and cpus. enable the cgroups and then check the configuration of your node. ---
**Author:** ![wakaka](https://avatars.discourse-cdn.com/v4/letter/w/d07c76/32.png) [@wakaka](https://community.openpbs.org/u/wakaka)\ **Post date:** [July 15, 2025, 6:51am UTC](https://community.openpbs.org/t/advanced-gpu-scheduling/4156/9 "2025-07-15T06:51:47Z")
Thank you very much for your reminder, Mr.adarsh. The memory problem is due to the setting of reserved\_amount, and the problem of the number of CPUs is due to hyperthreading.