I submitted a job and applied for 96 CPUs, which will run two programs in sequence. Program 1 is estimated to take 3 hours and use 96 CPUs, while Program 2 runs after Program 1 and is estimated to take 4 hours. However, it only requires 4 CPUs, which would result in resource waste. I found that the job dependency method can be used, but it needs to submit two jobs, and must first obtain the jobid of the first job. Is there a simpler way to handle this situation in only one job? Looking forward to any useful assistance.
No, there is no resize or altering of resource option/request that can be carried within the same job.
you have the correct option of using dependency chain.
jid1=$(qsub core96.pbs); jid2=$(qsub -W depend=afterok:$jid1 fourcore.pbs)
I think that if you enable allow_node_submit you can submit a job with qsub from within another job, so you could have your first job run “Program 1” then as its last action submit another job to run “Program 2”, however that’s a bit ‘dirty’ in my opinion. Doing what Adarsh suggested is cleaner. Running something like the following seems a pretty “simple[r] way to handle this situation”:
qsub -W depend=afterok:$(qsub program1.pbs) program2.pbs