I submitted a job and applied for 96 CPUs, which will run two programs in sequence. Program 1 is estimated to take 3 hours and use 96 CPUs, while Program 2 runs after Program 1 and is estimated to take 4 hours. However, it only requires 4 CPUs, which would result in resource waste. I found that the job dependency method can be used, but it needs to submit two jobs, and must first obtain the jobid of the first job. Is there a simpler way to handle this situation in only one job? Looking forward to any useful assistance.
No, there is no resize or altering of resource option/request that can be carried within the same job.
you have the correct option of using dependency chain.
jid1=$(qsub core96.pbs); jid2=$(qsub -W depend=afterok:$jid1 fourcore.pbs)
I think that if you enable allow_node_submit you can submit a job with qsub from within another job, so you could have your first job run “Program 1” then as its last action submit another job to run “Program 2”, however that’s a bit ‘dirty’ in my opinion. Doing what Adarsh suggested is cleaner. Running something like the following seems a pretty “simple[r] way to handle this situation”:
qsub -W depend=afterok:$(qsub program1.pbs) program2.pbs
I’d be happy to help, but I notice the thread content appears to be incomplete. The original post and recent replies sections are empty, so I can’t see what specific resource waste issue you’re asking about or what the two programs are.
Could you share a bit more detail about what you’re experiencing. For example:
- What type of programs are running together (databases, services, processes)?
- What resources are being wasted (CPU, memory, disk I/O, network bandwidth)?
- What’s your current system setup (shared server, containerized environment, local machine)?
- Have you noticed any specific metrics or symptoms that made you realize there’s waste happening?
Once I understand the actual problem, I can give you more targeted advice. Common culprits in these situations usually involve lock contention, duplicate caching, competing I/O operations, or inefficient resource scheduling, but the solution really depends on what you’re dealing with.
If you’re looking to consolidate or optimize two programs running in the same job, there are usually several angles to explore: process isolation levels, resource quotas, scheduling priorities, or architectural changes. But knowing more specifics would help me point you toward the most relevant solutions.
Feel free to paste the original problem details and I’ll take another look.
Thank you for your patient reply, Mr.miajohan79. The assignment involves two Python programs running sequentially. The first program processes some files in parallel and generates new files in a specified format. The second program then processes the newly generated files and generates a report. This second program runs serially using only one CPU.
When I put them both into one job, because the first program can run in parallel, it allocates multiple CPUs (e.g., 24). I didn’t explicitly allocate memory. When the first program finishes executing and the second program starts, it only uses one CPU. This results in wasted CPU time, depending on the execution time of the second program (e.g., 23 CPUs are idle at this time).
My current system is configured with local machine and NFS sharing. In fact, using job dependencies can indeed solve this problem, but it requires manually submitting two jobs, and it is necessary to ensure that the second job can access the files generated by the first job. Perhaps I can achieve this through a script.