# Regarding hard binding of resources

**URL:** <https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283>\
**Category:** Users/Site Administrators\
**Created:** [September 19, 2020, 3:56am UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283 "2020-09-19T03:56:20Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![narayan](https://avatars.discourse-cdn.com/v4/letter/n/22d042/32.png) [@narayan](https://community.openpbs.org/u/narayan)\
**Post date:** [September 19, 2020, 3:56am UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/1 "2020-09-19T03:56:20Z")

</div>

Please help me on below issue. (attached file of configuration)

**Point 1** - we have 7 old compute nodes with 24 cores each, and OpenPBS version 14.2 is installed. We have created 4 queues and for two groups that’s agarwal and indranil, for agarwal group we already configured 96 cores and for indranil group 72 cores configured, in pbs server level.

**Point 2 -** Now we added 8 new compute nodes on cluster with 32 cores on each node, that’s only related to agarwal group and we created new 3 queues for agarwal group.

**My question is,**

a) How to configure in pbs that agarwal group users submit job in new queues, and it should goes to new nodes only. (without binding the nodes to queue is possible? if node binding is necessary in this condition then how to bind all new nodes to all the new queues?)

b) How to configure in pbs that agarwal group users submit job in old queue with till 96 cores only? it should go to on old nodes because agarwal group limit is 96 cores as per old policy.

i.e How to set hard limit in 4 old queues that agarwal group users only use 96 cores only?

# Output of qmgr ‘p s’ for reference.

# 

# Create queues and set their attributes.

# 

# 

# Create and define queue mini

# 

create queue mini  
set queue mini queue\_type = Execution  
set queue mini resources\_max.ncpus = 48  
set queue mini resources\_max.walltime = 24:00:00  
set queue mini resources\_min.ncpus = 1  
set queue mini acl\_group\_enable = True  
set queue mini acl\_groups = Agarwal  
set queue mini acl\_groups += Indranil  
set queue mini enabled = True  
set queue mini started = True

# 

# Create and define queue long

# 

create queue long  
set queue long queue\_type = Execution  
set queue long resources\_max.ncpus = 24  
set queue long resources\_max.walltime = 120:00:00  
set queue long resources\_min.ncpus = 1  
set queue long acl\_group\_enable = True  
set queue long acl\_groups = Agarwal  
set queue long acl\_groups += Indranil  
set queue long enabled = True  
set queue long started = True

# 

# Create and define queue extralong

# 

create queue extralong  
set queue extralong queue\_type = Execution  
set queue extralong resources\_max.ncpus = 12  
set queue extralong resources\_max.walltime = 336:00:00  
set queue extralong resources\_min.ncpus = 1  
set queue extralong acl\_group\_enable = True  
set queue extralong acl\_groups = Agarwal  
set queue extralong acl\_groups += Indranil  
set queue extralong max\_run = [u:PBS\_GENERIC=1]  
set queue extralong enabled = True  
set queue extralong started = True

# 

# Create and define queue short

# 

create queue short  
set queue short queue\_type = Execution  
set queue short resources\_max.ncpus = 96  
set queue short resources\_max.walltime = 72:00:00  
set queue short resources\_min.ncpus = 1  
set queue short acl\_group\_enable = True  
set queue short acl\_groups = Agarwal  
set queue short acl\_groups += Indranil  
set queue short enabled = True  
set queue short started = True

# 

# Create and define queue smalln

# 

create queue smalln  
set queue smalln queue\_type = Execution  
set queue smalln resources\_max.ncpus = 128  
set queue smalln resources\_max.walltime = 48:00:00  
set queue smalln resources\_min.ncpus = 1  
set queue smalln acl\_group\_enable = True  
set queue smalln acl\_groups = Agarwal  
set queue smalln enabled = True  
set queue smalln started = True

# 

# Create and define queue longn

# 

create queue longn  
set queue longn queue\_type = Execution  
set queue longn resources\_max.ncpus = 64  
set queue longn resources\_max.walltime = 96:00:00  
set queue longn resources\_min.ncpus = 1  
set queue longn acl\_group\_enable = True  
set queue longn acl\_groups = Agarwal  
set queue longn enabled = True  
set queue longn started = True

# 

# Create and define queue extralongn

# 

create queue extralongn  
set queue extralongn queue\_type = Execution  
set queue extralongn resources\_max.ncpus = 64  
set queue extralongn resources\_max.walltime = 144:00:00  
set queue extralongn resources\_min.ncpus = 1  
set queue extralongn acl\_group\_enable = True  
set queue extralongn acl\_groups = Agarwal  
set queue extralongn enabled = True  
set queue extralongn started = True

# 

# Set server attributes.

# 

set server scheduling = True  
set server max\_run\_res.ncpus = [g:Agarwal=96]  
set server max\_run\_res.ncpus += [g:Indranil=72]  
set server max\_run\_res.ncpus += [u:PBS\_GENERIC=96]  
set server log\_events = 511  
set server mail\_from = adm  
set server query\_other\_jobs = True  
set server resources\_default.ncpus = 1  
set server default\_chunk.ncpus = 1  
set server scheduler\_iteration = 600  
set server resv\_enable = True  
set server node\_fail\_requeue = 310  
set server max\_array\_size = 10000  
set server pbs\_license\_min = 0  
set server pbs\_license\_max = 2147483647  
set server pbs\_license\_linger\_time = 31536000  
set server license\_count = Avail\_Global:1000000 Avail\_Local:1000000 Used:0 High\_Use:0 Avail\_Sockets:1000000 Unused\_Sockets:1000000  
set server eligible\_time\_enable = False  
set server max\_concurrent\_provision = 5

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [September 20, 2020, 6:29pm UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/2 "2020-09-20T18:29:10Z")

</div>

> [@narayan](#):
>
> a) How to configure in pbs that agarwal group users submit job in new queues, and it should goes to new nodes only. (without binding the nodes to queue is possible? if node binding is necessary in this condition then how to bind all new nodes to all the new queues?)

```
* qmgr -c "create resource group_name type=string_array,type=h"
* qmgr -c "set queue QUEUENAME default_chunk.group_name=agarwal"
* edit $PBS_HOME/sched_priv/sched_config 
     - find the resources: "ncpus, aoe.......,group_name" , append group_name to the end of the line
     - kill -HUP < PID of the pbs_sched process >
* for i in "list of agarwal nodes"; do qmgr -c "set node $i resources_available.group_name=agarwal";done

```

> [@narayan](#):
>
> b) How to configure in pbs that agarwal group users submit job in old queue with till 96 cores only? it should go to on old nodes because agarwal group limit is 96 cores as per old policy.

Same as above, but use different group\_name assigned to the default\_chunk of the queue.  
And nodes should be appended with that group\_name  
qmgr -c “set node NODENAME resources\_available+=agarwalold” # note += in this line to append

> [@narayan](#):
>
> i.e How to set hard limit in 4 old queues that agarwal group users only use 96 cores only?

queue limits can be used - please follow this section from the PBS Pro admin guide - Examples of Setting Server and Queue Limits

---

<div class="post-metadata">

**Author:** ![narayan](https://avatars.discourse-cdn.com/v4/letter/n/22d042/32.png) [@narayan](https://community.openpbs.org/u/narayan)\
**Post date:** [September 28, 2020, 4:27am UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/3 "2020-09-28T04:27:38Z")

</div>

> [@adarsh](#):
>
> `qmgr -c "create resource group_name type=string_array,type=h"`

Hi Adarsh,  
What you have suggested command is not working.

[root@gananamaster ~]# qmgr -c “create resource group\_name type=string\_array,type=h”  
qmgr obj=group\_name svr=default: Illegal attribute or resource value  
qmgr: Error (15014) returned from server

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [September 28, 2020, 5:05am UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/4 "2020-09-28T05:05:09Z")

</div>

> [@adarsh](#):
>
> qmgr -c “create resource group\_name type=string\_array,type=h”

Sorry, correction  
`qmgr -c "create resource group_name type=string_array,flag=h"`

---

<div class="post-metadata">

**Author:** ![narayan](https://avatars.discourse-cdn.com/v4/letter/n/22d042/32.png) [@narayan](https://community.openpbs.org/u/narayan)\
**Post date:** [September 28, 2020, 5:33am UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/5 "2020-09-28T05:33:47Z")

</div>

> [@adarsh](#):
>
> queue limits can be used - please follow this section from the PBS Pro admin guide - Examples of Setting Server and Queue Limits

Dear Adarsh,

Thank you for your reply.

I have checked the pbspro admin guide and I found there single queue level limit can be apply only.

But I need to bind my old 4 queues (mini,short,extralong,long) should have combine limit at  
group or user level.

Example  
In old 4 queues (mini,short,extralong,long) combine queue limit for agarwal group should be 96 core.

Please let me know in case any method apply the same.

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [September 28, 2020, 7:07pm UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/6 "2020-09-28T19:07:20Z")

</div>

You are welcome !

You can set the limit at the server level for users:

**Case1** :  
Generic Users = 4  
pbsuser01 = 2  
pbsuser02 = 6  
qmgr: set server max\_run +="[u:PBS\_GENERIC=4], [u:pbsuser01=2],[u:pbsuser02=6]"

**Case2** :  
Limit agarwal group = 6  
qmgr: set server max\_run = “[g:agarwal=6]”

Please check the server and queue limits chapter in the PS Pro admin guide.  
Hope this is helpful.

---

<div class="post-metadata">

**Author:** ![narayan](https://avatars.discourse-cdn.com/v4/letter/n/22d042/32.png) [@narayan](https://community.openpbs.org/u/narayan)\
**Post date:** [October 20, 2020, 4:56pm UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/7 "2020-10-20T16:56:00Z")

</div>

Dear Adarsh,

Please help me on belwo issue.

We have 15 compute nodes.  
We have configured 7 queue small, long, extralong, mini, smalln, longn and extralongn.  
smalln, longn, extralongn queues are binding with 8 nodes. When you submit job in smalln, longn, extralong queue job goes to binding nodes.

My question is

qstat command not showing jobs running time in smalln, longn, extralongn queue other  
queue is showing perfectly job running time.

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [October 20, 2020, 7:38pm UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/8 "2020-10-20T19:38:04Z")

</div>

> [@narayan](#):
>
> qstat command not showing jobs running time in smalln, longn, extralongn queue other  
> queue is showing perfectly job running time.

Could you please try these commands  
qstat -answ1  
qstat -T  
qstat -fx

---

<div class="post-metadata">

**Author:** ![narayan](https://avatars.discourse-cdn.com/v4/letter/n/22d042/32.png) [@narayan](https://community.openpbs.org/u/narayan)\
**Post date:** [October 21, 2020, 6:51am UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/9 "2020-10-21T06:51:47Z")

</div>

Dear Adarsh,

Thank you for your reply.

qstat -answ1  
qstat -T  
qstat -fx

all command working fine but only qstat is not showing, do I need to do any configuration  
that qstat will show the running time of jobs.

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [October 21, 2020, 10:23am UTC](https://community.openpbs.org/t/regarding-hard-binding-of-resources/2283/10 "2020-10-21T10:23:15Z")

</div>

Thank you @narayan

Could you please share the output of the above commands with obfuscation?  
Do all jobs request walltime ?