Facing Some Issue with Submitting Jobs on OpenPBS Cluster!

Hello everyone,

I am encountering an issue while trying to submit jobs to our OpenPBS cluster…, and I need some help figuring out what is going wrong. When I try to submit a job using the qsub command…, I get the error message: “Job submission failed: Resource temporarily unavailable.”

I have checked the following:

The cluster seems to be up and running (checked with pbsnodes).
The job script is correctly formatted, with all required resources specified.
Other users can submit jobs, so it’s not a general cluster issue.

I have also looked through the logs but haven’t found anything that indicates what the specific problem might be. I am not sure if this is related to a queue issue or a misconfiguration.

Has anyone faced a similar issue: ?? Any suggestions on how to troubleshoot or resolve this: ?? Would greatly appreciate any pointers or advice from those who have worked with OpenPBS extensively. I have also gone through this thread https://community.openpbs.org/t/issue-with-submitting-a-job-in-open-pbs-qlik-sense but still looking for some more tips and advice.

Thanks in advance for your help !!

With Regards,
Marcelo Salas

Please check and share

  • version of PBS Pro you are running, whether its services are up and running
  • check for space condition
  • is there any limit on lsof or any limitation on net.ipv4.tcp_retries2
  • in $PBS_HOME/sched_priv/sched_config , set provisioning to below instead of agressive_provision and kill -HUP
    provision_policy: “avoid_provision”