# About vnode creation for each socket

**URL:** https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935
**Category:** Users/Site Administrators
**Created:** [January 10, 2020, 12:13am UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935 "2020-01-10T00:13:42Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![hiromi](https://avatars.discourse-cdn.com/v4/letter/h/bc79bd/32.png) [@hiromi](https://community.openpbs.org/u/hiromi)
#### Post date: [January 10, 2020, 12:13am UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935/1 "2020-01-10T00:13:42Z")

</div>

Dear All,

Can I create a vnode by specifying a physical CPU (socket)?  
I ’d like to execute different types of jobs for each CPU socket.  
If you have a way to control jobs on a per-socket basis other than creating a vnode, that’s fine.  
Let us know if you have any good ideas.

---

<div class="post-metadata">

### Author: ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)
#### Post date: [January 10, 2020, 10:07pm UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935/2 "2020-01-10T22:07:23Z")

</div>

Please check cgroups discussion topic on this forum.

- Implementation of cgroups might be helpful in isolating the sockets for the job(s).
- core pinning using MPI flavours would be help

It would be helpful to know the kind of jobs or applications you would like to run on the specific socket(s) and the reason behind it ?

---

<div class="post-metadata">

### Author: ![hiromi](https://avatars.discourse-cdn.com/v4/letter/h/bc79bd/32.png) [@hiromi](https://community.openpbs.org/u/hiromi)
#### Post date: [January 14, 2020, 2:21am UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935/3 "2020-01-14T02:21:58Z")

</div>

Thanks for the advice.

The background of this question is as follows.  
The question is focused on servers with AMD CPU (2 sockets) and GPU.

- Multiple GPUs are connected to CPU1 by PCI gen4
- GPU is not connected to CPU2  
In that case, CPU1 is used for machine learning and CPU2 is used for simulation for JOB  
It is a question whether you can use it.

We have already implemented GPU control using cgroups hooks.

I checked [2-cpu executor host](http://community.openpbs.org/t/2-cpu-executor-host/322).  
When controlling with a socket, do I need to make my own hook?  
If possible, we ’d like to to separate the job types by queue, so we thought it would be nice to be able to control at the vnode level.

Thanks,

---

<div class="post-metadata">

### Author: ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)
#### Post date: [January 14, 2020, 9:46am UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935/4 "2020-01-14T09:46:22Z")

</div>

Thank you @hiromi

> [@hiromi](#):
>
> When controlling with a socket, do I need to make my own hook?

1/ Yes , it would be best option  
2/ otherwise, map the topology of the system to your cgroup hook.( not sure about the complexity)

> [@hiromi](#):
>
> If possible, we ’d like to to separate the job types by queue, so we thought it would be nice to be able to control at the vnode level.

The job type can also be part of a hook or it can be part of the Qlist configuration ([Reserve resources for a user for particular Job within queue when using Node binding - #2 by adarsh](http://community.openpbs.org/t/reserve-resources-for-a-user-for-particular-job-within-queue-when-using-node-binding/1657/2)) , so that respective vnode can be targetted

Thank you,

---

<div class="post-metadata">

### Author: ![hiromi](https://avatars.discourse-cdn.com/v4/letter/h/bc79bd/32.png) [@hiromi](https://community.openpbs.org/u/hiromi)
#### Post date: [January 15, 2020, 6:13am UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935/5 "2020-01-15T06:13:22Z")

</div>

Thank you Adarsh,

If possible, we 'd like to implement it in a way that does not create a new hook.  
What does mapping to a cgroup hook mean? Is it possible to define it in pbs\_cgroups.py? Or do you add some definition to the OS side cpuset?

Also, I hope that you can control with Qlist, but for that, I think that you need a vnode defined for each socket. Does this mean that we can’t do this right now?

Thank you.

---

<div class="post-metadata">

### Author: ![hiromi](https://avatars.discourse-cdn.com/v4/letter/h/bc79bd/32.png) [@hiromi](https://community.openpbs.org/u/hiromi)
#### Post date: [January 17, 2020, 5:31am UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935/6 "2020-01-17T05:31:39Z")

</div>

I have additional questions.  
In this case, if vnode\_per\_numa\_node is enabled in cgroups hook configuration, will a vnode be created for each socket?

Thank you.

---

<div class="post-metadata">

### Author: ![mkaro](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/mkaro/32/85_2.png) [@mkaro](https://community.openpbs.org/u/mkaro)
#### Post date: [January 17, 2020, 7:52pm UTC](https://community.openpbs.org/t/about-vnode-creation-for-each-socket/1935/7 "2020-01-17T19:52:46Z")

</div>

That is correct. You may find additional information about the cgroup hook in chapter 15 of the PBS Pro Admin Guide. [https://www.altair.com/pdfs/pbsworks/PBSAdminGuide19.2.3.pdf](https://www.altair.com/pdfs/pbsworks/PBSAdminGuide19.2.3.pdf)
