About vnode creation for each socket

Users/Site Administrators
1

Dear All,

Can I create a vnode by specifying a physical CPU (socket)?
I ’d like to execute different types of jobs for each CPU socket.
If you have a way to control jobs on a per-socket basis other than creating a vnode, that’s fine.
Let us know if you have any good ideas.

2

Please check cgroups discussion topic on this forum.

  • Implementation of cgroups might be helpful in isolating the sockets for the job(s).
  • core pinning using MPI flavours would be help

It would be helpful to know the kind of jobs or applications you would like to run on the specific socket(s) and the reason behind it ?

3

Thanks for the advice.

The background of this question is as follows.
The question is focused on servers with AMD CPU (2 sockets) and GPU.

  • Multiple GPUs are connected to CPU1 by PCI gen4
  • GPU is not connected to CPU2
    In that case, CPU1 is used for machine learning and CPU2 is used for simulation for JOB
    It is a question whether you can use it.

We have already implemented GPU control using cgroups hooks.

I checked 2-cpu executor host.
When controlling with a socket, do I need to make my own hook?
If possible, we ’d like to to separate the job types by queue, so we thought it would be nice to be able to control at the vnode level.

Thanks,

4

Thank you @hiromi

1/ Yes , it would be best option
2/ otherwise, map the topology of the system to your cgroup hook.( not sure about the complexity)

The job type can also be part of a hook or it can be part of the Qlist configuration (Reserve resources for a user for particular Job within queue when using Node binding - #2 by adarsh) , so that respective vnode can be targetted

Thank you,

5

Thank you Adarsh,

If possible, we 'd like to implement it in a way that does not create a new hook.
What does mapping to a cgroup hook mean? Is it possible to define it in pbs_cgroups.py? Or do you add some definition to the OS side cpuset?

Also, I hope that you can control with Qlist, but for that, I think that you need a vnode defined for each socket. Does this mean that we can’t do this right now?

Thank you.

6

I have additional questions.
In this case, if vnode_per_numa_node is enabled in cgroups hook configuration, will a vnode be created for each socket?

Thank you.

7

That is correct. You may find additional information about the cgroup hook in chapter 15 of the PBS Pro Admin Guide. https://www.altair.com/pdfs/pbsworks/PBSAdminGuide19.2.3.pdf

1 Like