# Qsub permission dennied **URL:** \ **Category:** Users/Site Administrators\ **Created:** [July 27, 2018, 2:21am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095 "2018-07-27T02:21:59Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [July 27, 2018, 2:22am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/1 "2018-07-27T02:22:00Z")
Hi Team, I faced a very wired issue. qsub is able to submit job but the job always failed per stderr oupt. could you please share how to debug this issue ? Thanks. zxhu@host:~$ echo ‘hostname’|qsub -q lnx64 100.cictest-09 zxhu@host ~$ cat STDIN.e100 -bash: line 1: /var/spool/pbs/mom\_priv/jobs/100.cictest-09.SC: Permission denied ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [July 27, 2018, 7:07am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/2 "2018-07-27T07:07:09Z")
Please check/share the below information , you might be able to trace the issue - tracejob 100 - on the compute node where this job ran, check the mom logs ( $PBS\_HOME/mom\_priv/mom\_logs/YYYYMMDD ) ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [July 27, 2018, 7:58am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/3 "2018-07-27T07:58:24Z")
Thanks for reply. this is the cmd I submit the job zxhu@sjdpc-zxhu:~$ echo ‘hostname’|qsub -q lnx64 131.cictest-09 zxhu@sjdpc-zxhu:~$ tracejob 131 Job: 131.cictest-09 07/27/2018 06:51:06 S enqueuing into lnx64, state 1 hop 1 07/27/2018 06:51:06 S Job Queued at request of zxhu@sjdpc-zxhu., owner = zxhu@sjdpc-zxhu., job name = STDIN, queue = lnx64 07/27/2018 06:51:07 L Considering job to run 07/27/2018 06:51:07 S Job Run at request of Scheduler@cictest-09 on exec\_vnode (cictest-04:ncpus=1) 07/27/2018 06:51:07 S Job Modified at request of Scheduler@cictest-09 07/27/2018 06:51:07 S Exit\_status=126 resources\_used.cpupercent=0 resources\_used.cput=00:00:00 resources\_used.mem=0kb resources\_used.ncpus=1 resources\_used.vmem=0kb resources\_used.walltime=00:00:00 07/27/2018 06:51:07 L Job run 07/27/2018 06:51:07 S Obit received momhop:1 serverhop:1 state:4 substate:42 this is the mom\_logs 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 464, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 464, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 464, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 463, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0100;pbs\_mom;Req;;Type 1 request received from root@172.19.197.97:15001, sock=1 07/27/2018 08:52:12;0100;pbs\_mom;Req;;Type 3 request received from root@172.19.197.97:15001, sock=1 07/27/2018 08:52:12;0100;pbs\_mom;Req;;Type 5 request received from root@172.19.197.97:15001, sock=1 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 464, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0008;pbs\_mom;Job;131.cictest-09;Started, pid = 5370 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 464, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0080;pbs\_mom;Job;131.cictest-09;task 00000001 terminated 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 463, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0008;pbs\_mom;Job;131.cictest-09;Terminated 07/27/2018 08:52:12;0100;pbs\_mom;Job;131.cictest-09;task 00000001 cput= 0:00:00 07/27/2018 08:52:12;0008;pbs\_mom;Job;131.cictest-09;kill\_job 07/27/2018 08:52:12;0100;pbs\_mom;Job;131.cictest-09;cictest-04 cput= 0:00:00 mem=0kb 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 464, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0008;pbs\_mom;Job;131.cictest-09;no active tasks 07/27/2018 08:52:12;0100;pbs\_mom;Job;131.cictest-09;Obit sent 07/27/2018 08:52:12;0100;pbs\_mom;Req;;Type 54 request received from root@172.19.197.97:15001, sock=1 07/27/2018 08:52:12;0080;pbs\_mom;Job;131.cictest-09;copy file request received 07/27/2018 08:52:12;0100;pbs\_mom;Job;131.cictest-09;staged 2 items out over 0:00:00 07/27/2018 08:52:12;0800;pbs\_mom;n/a;mom\_get\_sample;nprocs: 464, cantstat: 0, nomem: 0, skipped: 0, cached: 0, max excluded PID: 0 07/27/2018 08:52:12;0008;pbs\_mom;Job;131.cictest-09;no active tasks 07/27/2018 08:52:12;0100;pbs\_mom;Req;;Type 6 request received from root@172.19.197.97:15001, sock=1 07/27/2018 08:52:12;0080;pbs\_mom;Job;131.cictest-09;delete job request received 07/27/2018 08:52:12;0008;pbs\_mom;Job;131.cictest-09;kill\_job ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [July 27, 2018, 8:23am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/4 "2018-07-27T08:23:50Z")
> [@squallabc](#): > > 07/27/2018 06:51:07 S **Exit\_status=126** resources\_used.cpupercent=0 resources\_used.cput=00:00:00 resources\_used.mem=0kb resources\_used.ncpus=1 resources\_used.vmem=0kb Exit\_Status is non-zero Exit code - 126 – Command invoked cannot execute The command “hostname” might not be able to execute on that compute node by that user, try echo “/bin/hostname” | qsub -q lnx64 ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [July 27, 2018, 2:31pm UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/5 "2018-07-27T14:31:47Z")
still the same error. could you please advise ? Thanks. zxhu@sjdpc-zxhu:~$ echo ‘/bin/hostname’|qsub -V -q lnx64 193.cictest-09 zxhu@sjdpc-zxhu:~$ cat STDIN.e193 -bash: line 1: /var/spool/pbs/mom\_priv/jobs/193.cictest-09.SC: Permission denied ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [July 30, 2018, 7:31am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/6 "2018-07-30T07:31:25Z")
> [@squallabc](#): > > -bash: line 1: /var/spool/pbs/mom\_priv/jobs/193.cictest-09.SC: Permission denied 1. Please check password-less SSH is working for all the users (seamless without Stricthostkey checking) - ssh from server to mom should be seamless \* ssh from mom to server should be seamless \* ssh between the Mom’s should be seamless 1. Please add the below lines in the same order in /etc/pbs.conf on the Server and Compute nodes and restart the services PBS\_RCP=/bin/false PBS\_SCP=/usr/bin/scp PBS\_RSHCOMMAND=/usr/bin/ssh 2. qmgr -c “set server flatuid=true” 3. Try running an interactive job as below qsub -l select=1:ncpus=1 -I (last argument to qsub here is -l is whichi is hypen capital I , i for Icecream) ---
**Author:** ![Source](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/source/32/756_2.png) [@Source](https://community.openpbs.org/u/Source)\ **Post date:** [September 14, 2018, 5:12am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/7 "2018-09-14T05:12:57Z")
Just a remind, i doubt if the $HOME directory of this user exists. if not, the job would failed as soon as it ran. I can’t remember exactly the error is, but suggest you to do a quick check. ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [September 22, 2018, 1:19pm UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/8 "2018-09-22T13:19:54Z")
Hi Adarsh, sorry for late response. the interactive job is okay no matter flatuid set or not zxhu@master:~$ qsub -l select=1:ncpus=1 -I qsub: waiting for job 534.master to start qsub: job 534.master ready Login on Linux, source .bashrc profile zxhu@node1:~$ but if it is failed with the same error msg when submitting a batch job. -bash: line 1: /var/spool/pbs/mom\_priv/jobs/533.sjolmbench.SC: Permission denied ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [September 22, 2018, 1:20pm UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/9 "2018-09-22T13:20:34Z")
Thanks . the $HOME exists. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [September 22, 2018, 6:58pm UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/10 "2018-09-22T18:58:05Z")
> [@squallabc](#): > > -bash: line 1: /var/spool/pbs/mom\_priv/jobs/533.sjolmbench.SC: Permission denied Please share the permissions of /var/spool/pbs/mom\_priv ( after when an interactive job is running). The permission on my system is as below: **[root@compute\_node ~]# namei -mo /var/spool/pbs/spool/** f: /var/spool/pbs/spool/ dr-xr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root spool drwxr-xr-x root root pbs drwxrwxrwt root root spool **[root@compute\_node mom\_priv]# namei -mo /var/spool/pbs/mom\_priv/jobs** f: /var/spool/pbs/mom\_priv/jobs dr-xr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root spool drwxr-xr-x root root pbs drwxr-x–x root root mom\_priv drwxr-x–x root root jobs ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [September 25, 2018, 9:11am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/11 "2018-09-25T09:11:23Z")
root@node1:~# namei -mo /var/spool/pbs/spool/ f: /var/spool/pbs/spool/ dr-xr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root spool drwxr-xr-x root root pbs drwxrwxrwt root root spool root@node1:~# namei -mo /var/spool/pbs/mom\_priv/jobs/ f: /var/spool/pbs/mom\_priv/jobs/ dr-xr-xr-x root root / drwxr-xr-x root root var drwxr-xr-x root root spool drwxr-xr-x root root pbs drwxr-x— root root mom\_priv drwxr-xr-x root root jobs and I find the file looks like below, it is an interactive job, is it correct ? -rwx------ 1 zxhu group1 22 Sep 25 01:43 372085.master1.SC Thanks. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [September 25, 2018, 9:32am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/12 "2018-09-25T09:32:01Z")
Thank you , could you please share the contents of the .SC file. ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [September 26, 2018, 1:53am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/13 "2018-09-26T01:53:17Z")
#!/bin/bash sleep 1000 Thanks. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [September 26, 2018, 7:21am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/14 "2018-09-26T07:21:45Z")
Can you please try this and check whether they run : - qsub – /bin/hostname - qsub – /bin/sleep 10 I am not sure whether you can run the same script by removing #! line ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [September 26, 2018, 8:01am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/15 "2018-09-26T08:01:52Z")
Thanks. if I remove ‘#!/bin/bash’ , it seems to be work. do you know why ? ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [September 26, 2018, 8:04am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/16 "2018-09-26T08:04:02Z")
> [@squallabc](#): > > do you know why ? Can you run that script as that user without using PBS on that compute node ? ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [September 26, 2018, 8:20am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/17 "2018-09-26T08:20:37Z")
it is able to run without any issue if I run the script without PBS. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [September 26, 2018, 8:22am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/18 "2018-09-26T08:22:20Z")
Strange, then please try this qsub -S /bin/bash - - sleep.sh ---
**Author:** ![squallabc](https://avatars.discourse-cdn.com/v4/letter/s/898d66/32.png) [@squallabc](https://community.openpbs.org/u/squallabc)\ **Post date:** [September 26, 2018, 8:27am UTC](https://community.openpbs.org/t/qsub-permission-dennied/1095/19 "2018-09-26T08:27:35Z")
execute script mode always works. ☹ the issue only happens on submitting from STDIN mode and the node is RHEL6 or later. there is no issue on RHEL5.