# Qsub set maximum number of job by me

**URL:** <https://community.openpbs.org/t/qsub-set-maximum-number-of-job-by-me/1567>\
**Category:** Users/Site Administrators\
**Created:** [April 25, 2019, 7:08am UTC](https://community.openpbs.org/t/qsub-set-maximum-number-of-job-by-me/1567 "2019-04-25T07:08:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rudrab](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/rudrab/32/201_2.png) [@rudrab](https://community.openpbs.org/u/rudrab)\
**Post date:** [April 25, 2019, 7:08am UTC](https://community.openpbs.org/t/qsub-set-maximum-number-of-job-by-me/1567/1 "2019-04-25T07:08:36Z")

</div>

Dear all,  
I am submitting a number of jobs using for loop via python as:

```
for x in X:
    for a in A:
      for m in M:
     sample = "-".join([m,a,x])
      os.chdir(sample)
      os.system(qsub script.sh)

```

The problem is in our local cluster implementation, we don’t have a `nice`, and jobs are submitted in _first come first serve_ basis. But, at any time, if my usage goes beyond 20% of nodes, sysadmin can kill it. I understand that this is not an optimal implementation of `qsub`, but this is of course beyond my control.

I currently solve this problem by limiting my jobs as:

```
while True:
  numjobs = os.system("qselect |wc -l).read()
  if numjobs <4:
   break
  else:
    time.sleep(30)

```

Which is not very decent, but serves the purpose.

My question is, is there any way to tell `qsub` how many jobs I can have at any given time so that I don’t need to check via `qselect` on each loop?

---

<div class="post-metadata">

**Author:** ![visheshh](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/visheshh/32/209_2.png) [@visheshh](https://community.openpbs.org/u/visheshh)\
**Post date:** [April 25, 2019, 8:38am UTC](https://community.openpbs.org/t/qsub-set-maximum-number-of-job-by-me/1567/2 "2019-04-25T08:38:52Z")

</div>

@rudrab You can set a user limit for max\_queued or max\_run on server.

---

<div class="post-metadata">

**Author:** ![rudrab](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/rudrab/32/201_2.png) [@rudrab](https://community.openpbs.org/u/rudrab)\
**Post date:** [April 26, 2019, 12:29pm UTC](https://community.openpbs.org/t/qsub-set-maximum-number-of-job-by-me/1567/3 "2019-04-26T12:29:40Z")

</div>

Thanks a lot for your reply.  
The PBSpro admin guide says:

```
max_run
The maximum number of jobs that can be running.
max_queued
The maximum number of jobs that can be queued and running. At the server level, this includes all jobs in the
complex. Queueing a job includes the qsub and qmove commands and the equivalent APIs.

```

I am not sure what will happen if my jobs hit `max_run`, but as shown in my original post, I have more jobs to submit through the for loop. Will qsub wait patiently to submit the next job? or it will exit?

Kindly help.

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [April 26, 2019, 8:34pm UTC](https://community.openpbs.org/t/qsub-set-maximum-number-of-job-by-me/1567/4 "2019-04-26T20:34:41Z")

</div>

> [@rudrab](#):
>
> My question is, is there any way to tell `qsub` how many jobs I can have at any given time so that I don’t need to check via `qselect` on each loop?

Answer is No , qsub does not know or have the blueprint of the existing status of the cluster, it is only a job submission client. The blueprint of the system is maintained by the server and using this blueprint scheduler decides where to schedule the job based on the scheduling policy/limits/sorting etc.

It seems, you want to have a automatic counter active measure (gaming the system) to submit jobs keeping in mind the policy set by your administrator of killing jobs if user jobs threshold is above 20%.  
It is almost like writing a scheduler for a scheduler .

> [@rudrab](#):
>
> I am not sure what will happen if my jobs hit `max_run` , but as shown in my original post, I have more jobs to submit through the for loop

You can submit as many jobs as you like (via qsub) , number of jobs that would be running at a time would be equal to the value set in ‘max\_run’ , if it is set to 3, only 3 jobs can run, no matter you have 1000’s of jobs in the queue ( even if the resources are available for them to run, due to max\_run limit , only 3 or X jobs will be running)

> [@rudrab](#):
>
> Will qsub wait patiently to submit the next job? or it will exit?

qsub submits the job(s) , job(s) will be accepted by the server , they will be assigned a job id , but based on the limits, they will be put in the queue until they are eligible to run based on the limits set by the administrator.

qsub is a job submission client, it submits the job and would not wait for anything .

You can write a wrapper script (as you have it now) , which takes all the inputs from the user , creates a script, check whether you are below 20% threshold, if you are then submits the job, otherwise it will in a loop and check the eligibility.

I hope this helps