# How gpu\_id allocated to physical gpu?

**URL:** <https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382>\
**Category:** Users/Site Administrators\
**Created:** [December 9, 2016, 4:17am UTC](https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382 "2016-12-09T04:17:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![yuri](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/yuri/32/36_2.png) [@yuri](https://community.openpbs.org/u/yuri)\
**Post date:** [December 9, 2016, 4:17am UTC](https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382/1 "2016-12-09T04:17:24Z")

</div>

Hi, everyone.  
I try to manage GPU resources with PBS Pro.  
I configured node following [this tutorial](https://www.aravision.com/products/pbsworks/pdf/PBS_NVIDIA_GPU_Scheduling.pdf). Now node can allows both ngpus and gpu\_id.

One things I’d clarify is how each gpu\_id allocate to physical GPU ID.  
Let’s say. In my node, 2 card can be displaied with nvidia-smi.

GPU 0: Tesla K40m (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx)  
GPU 1: Tesla K40m (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx)

And then I set gpu0, gpu1 to MoM.  
In this case, below understanding is correct?

gpu0 ==(allocate)==\> GPU 0: Tesla K40m (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx)  
gpu1 ==(allocate)==\> GPU 1: Tesla K40m (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx)

Thank you in advance.

---

<div class="post-metadata">

**Author:** ![jon](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/jon/32/23_2.png) [@jon](https://community.openpbs.org/u/jon)\
**Post date:** [December 13, 2016, 3:28am UTC](https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382/2 "2016-12-13T03:28:43Z")

</div>

The tutorial that you followed allows you to limit the number of jobs that request gpus that can be placed on a given node. If you use the advanced configuration, then you can also see which gpu\_id is assigned to the job if your job parses the select line to see which gpu\_id was assigned.

Another option is to use the cgroup hook that was contributed to the community around August, it removes the need to use the gpu\_id. It will assign the gpus to the jobs cgroup. When the job is launched, the environment variable is set, via an execjob\_launch hook, for the job indicating that which gpu(s) to use.

---

<div class="post-metadata">

**Author:** ![yuri](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/yuri/32/36_2.png) [@yuri](https://community.openpbs.org/u/yuri)\
**Post date:** [December 13, 2016, 4:14am UTC](https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382/3 "2016-12-13T04:14:01Z")

</div>

Hi, jon.

Thank you for your reply and suggestion of cgroup option.  
I’m sorry that my explanation is difficult to understand.

I’ve already use the advanced configuration.  
And my concern is whether the gpu\_id which is set in advanced config corresponds to the GPU ID extracted from nvidia-smi or not.

it’s like:  
gpu0@PBS = GPU 0@nvidia-smi  
gpu1@PBS = GPU1@nvidia-smi

if yes, I can decide which gpu can be used with gpu\_id.

I would appriciate your advice.

---

<div class="post-metadata">

**Author:** ![jon](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/jon/32/23_2.png) [@jon](https://community.openpbs.org/u/jon)\
**Post date:** [December 13, 2016, 4:47am UTC](https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382/4 "2016-12-13T04:47:15Z")

</div>

Yuri,

There is no direct connection with the gpu\_id and the GPU. If the vnode is setup correctly then the scheduler will place the job on the node but there is no guarantee that the job will use the cores and gpu on the same socket. This just allows you to know which gpu the scheduler assigned and you can setup the job environment from there.

If you need the cores, memory, and gpus tied to the assigned socket then this is where cgroups come in. Cgroups will assign the cores, memory socket, and devices to the assigned socket(s) as scheduled by the scheduler

---

<div class="post-metadata">

**Author:** ![yuri](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/yuri/32/36_2.png) [@yuri](https://community.openpbs.org/u/yuri)\
**Post date:** [December 13, 2016, 5:02am UTC](https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382/5 "2016-12-13T05:02:07Z")

</div>

Hi, jon.  
I understand. So, cgoups hook is worth digging in my case.  
Thank you for your advice.

---

<div class="post-metadata">

**Author:** ![oscar](https://avatars.discourse-cdn.com/v4/letter/o/22d042/32.png) [@oscar](https://community.openpbs.org/u/oscar)\
**Post date:** [March 31, 2017, 7:23am UTC](https://community.openpbs.org/t/how-gpu-id-allocated-to-physical-gpu/382/6 "2017-03-31T07:23:06Z")

</div>

Hello

Once my job is running, how can I access the value of “gpu\_id” (or any other BPS resource value) for my own job?

Thank you  
Óscar