Working with .SC files

After having my IT support issue some new hostnames and updating all the configs and references to hostnames. I’m getting an error from /var/spool/pbs/mom_priv/jobs/.hostname.SC about PBS not being able to call Singularity.The job submission script (due to supporting existing job scripts used for a previous job scheduler) is rather complex.

I’m looking for some guidance on how to work with these .SC files or any other thoughts. I haven’t found much documentation and I’m not getting very far with the output from tracejob or qstat.

The cluster is using Ubuntu 18.04.


This is the job script, if in case jobscript was provided to qsub ( for example: qsub
The contents of this would be seen in .PBSSERVER.SC

Please check the stdout and stderr for this submission, it might give us some information.