# Job gettting submitted to only 01 node in 04 node cluster **URL:** \ **Category:** Users/Site Administrators\ **Created:** [March 22, 2019, 1:52am UTC](https://community.openpbs.org/t/job-gettting-submitted-to-only-01-node-in-04-node-cluster/1520 "2019-03-22T01:52:20Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![anieshmathew](https://avatars.discourse-cdn.com/v4/letter/a/9dc877/32.png) [@anieshmathew](https://community.openpbs.org/u/anieshmathew)\ **Post date:** [March 22, 2019, 1:52am UTC](https://community.openpbs.org/t/job-gettting-submitted-to-only-01-node-in-04-node-cluster/1520/1 "2019-03-22T01:52:20Z")
When I submit a job to a cluster of 04 nodes and 01 head node, job gets allotted to only one node. Is there any specific configuration that I could have left out? [hpcuser@mgt1 ~]$ qsub -I -l select=2:ncpus=12;mpiprocs=12 qsub: waiting for job 7.mgt1 to start qsub: job 7.mgt1 ready [hpcuser@mgt1 ~]$ qsub -I -l select=2:ncpus=12:mpiprocs=12 [hpcuser@mgt1 ~]$ qstat -an mgt1: Req’d Req’d Elap Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time * * * 1.mgt1 hpcuser workq STDIN 25701 2 20 – – R 01:26 node01/0_10+node01/1_10 7.mgt1 hpcuser workq STDIN 26470 2 24 – – R 00:00 node01/2_12+node01/3_12 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ The qstat -f of job id 1.mgt1 is below [root@mgt1 test]# qstat -f Job Id: 1.mgt1 Job\_Name = STDIN Job\_Owner = hpcuser@mgt1 resources\_used.cpupercent = 98 resources\_used.cput = 00:45:26 resources\_used.mem = 48141760kb resources\_used.ncpus = 20 resources\_used.vmem = 48904932kb resources\_used.walltime = 01:23:10 job\_state = R queue = workq server = mgt1 Checkpoint = u ctime = Thu Mar 21 18:34:41 2019 Error\_Path = /dev/pts/0 exec\_host = node01/0_10+node01/1_10 exec\_vnode = (node01:ncpus=10)+(node01:ncpus=10) Hold\_Types = n interactive = True Join\_Path = n Keep\_Files = n Mail\_Points = a mtime = Thu Mar 21 19:57:50 2019 Output\_Path = /dev/pts/0 Priority = 0 qtime = Thu Mar 21 18:34:41 2019 Rerunable = False Resource\_List.ncpus = 20 Resource\_List.nodect = 2 Resource\_List.place = free Resource\_List.select = 2:ncpus=10 schedselect = 2:ncpus=10 stime = Thu Mar 21 18:34:41 2019 session\_id = 25701 jobdir = /nfsshare/home/hpcuser substate = 42 Variable\_List = PBS\_O\_HOME=/nfsshare/home/hpcuser,PBS\_O\_LANG=en\_US.UTF-8, PBS\_O\_LOGNAME=hpcuser, PBS\_O\_PATH=/opt/xcat/bin:/opt/xcat/sbin:/opt/xcat/share/xcat/tools:/op t/hpc/ferret/ferret-7.4.4-RHEL7-64/bin:/opt/xcat/bin:/opt/xcat/sbin:/op t/xcat/share/xcat/tools:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sb in:/bin:/sbin:/opt/pbs/bin:/nfsshare/home/hpcuser/.local/bin:/nfsshare/ home/hpcuser/bin:/opt/pbs/bin,PBS\_O\_MAIL=/var/spool/mail/hpcuser, PBS\_O\_SHELL=/bin/bash,PBS\_O\_WORKDIR=/nfsshare/home/hpcuser, PBS\_O\_SYSTEM=Linux,PBS\_O\_QUEUE=workq,PBS\_O\_HOST=mgt1 euser = hpcuser egroup = hpcuser hashname = 1.mgt1 queue\_rank = 1553173481022 queue\_type = E comment = Job run at Thu Mar 21 at 18:34 on (node01:ncpus=10)+(node01:ncpus =10) etime = Thu Mar 21 18:34:41 2019 run\_count = 1 Submit\_arguments = -I -X -l select=2:ncpus=10 project = \_pbs\_project\_default forward\_x11\_cookie = MIT-MAGIC-COOKIE-1:39c4016e0ca635f02ef3b35c73e44524:0 forward\_x11\_port = True run\_version = 1 +++++++++++++++++++++++++++++++++++++++++++++++++++ The pbsnodes -a shows the nodes node details as below [root@mgt1 ~]# pbsnodes -a node01 Mom = node01 Port = 15002 pbs\_version = 19.1.1 ntype = PBS state = free pcpus = 48 resources\_available.arch = linux resources\_available.host = node01 resources\_available.mem = 65184884kb resources\_available.ncpus = 48 resources\_available.vnode = node01 resources\_assigned.accelerator\_memory = 0kb resources\_assigned.hbmem = 0kb resources\_assigned.mem = 0kb resources\_assigned.naccelerators = 0 resources\_assigned.ncpus = 0 resources\_assigned.vmem = 0kb resv\_enable = True sharing = default\_shared last\_state\_change\_time = Thu Mar 21 13:43:54 2019 node02 Mom = node02 Port = 15002 pbs\_version = 19.1.1 ntype = PBS state = free pcpus = 48 resources\_available.arch = linux resources\_available.host = node02 resources\_available.mem = 65184884kb resources\_available.ncpus = 48 resources\_available.vnode = node02 resources\_assigned.accelerator\_memory = 0kb resources\_assigned.hbmem = 0kb resources\_assigned.mem = 0kb resources\_assigned.naccelerators = 0 resources\_assigned.ncpus = 0 resources\_assigned.vmem = 0kb resv\_enable = True sharing = default\_shared last\_state\_change\_time = Thu Mar 21 13:43:54 2019 node03 Mom = node03 Port = 15002 pbs\_version = 19.1.1 ntype = PBS state = free pcpus = 48 resources\_available.arch = linux resources\_available.host = node03 resources\_available.mem = 65184884kb resources\_available.ncpus = 48 resources\_available.vnode = node03 resources\_assigned.accelerator\_memory = 0kb resources\_assigned.hbmem = 0kb resources\_assigned.mem = 0kb resources\_assigned.naccelerators = 0 resources\_assigned.ncpus = 0 resources\_assigned.vmem = 0kb resv\_enable = True sharing = default\_shared last\_state\_change\_time = Thu Mar 21 13:43:54 2019 node04 Mom = node04 Port = 15002 pbs\_version = 19.1.1 ntype = PBS state = free pcpus = 24 resources\_available.arch = linux resources\_available.host = node04 resources\_available.mem = 65185004kb resources\_available.ncpus = 24 resources\_available.vnode = node04 resources\_assigned.accelerator\_memory = 0kb resources\_assigned.hbmem = 0kb resources\_assigned.mem = 0kb resources\_assigned.naccelerators = 0 resources\_assigned.ncpus = 0 resources\_assigned.vmem = 0kb resv\_enable = True sharing = default\_shared last\_state\_change\_time = Thu Mar 21 13:43:54 2019 ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [March 22, 2019, 11:06am UTC](https://community.openpbs.org/t/job-gettting-submitted-to-only-01-node-in-04-node-cluster/1520/2 "2019-03-22T11:06:53Z")
Try submitting jobs as below: `qsub -I -l select=2:ncpus=12;mpiprocs=12 -l place=scatter` This place statement tells that you want 1 chunk to be taken from two different nodes. So it will now take 12 cores from one node and another 12 cores from another node. If you do not specify place, it will default to free/pack, you have 24 cores available on one system, hence both the chunks will be given/taken from one node I hope this works for you