PBS fails to change TMPDIR

Users/Site Administrators
1

Hi,

I am having some trouble to change the default tmpdir from /var/tmp to another location, such as /scratch/tmp (a different hard drive). I try editing either /etc/pbs.conf or mom_priv/config, or both, but none of them works. My pbs.conf:

PBS_SERVER=litio
PBS_START_SERVER=1
PBS_START_SCHED=1
PBS_START_COMM=1
PBS_START_MOM=1
PBS_EXEC=/opt/pbs
PBS_HOME=/var/spool/pbs
PBS_CORE_LIMIT=unlimited
PBS_SCP=/bin/scp
PBS_TMPDIR=/scratch/tmp

Delivers the error:

pbs_mom: Operation not permitted (1) in tmp_file_sec, Security violation “/scratch/tmp” resolves to “/scratch”
/opt/pbs/sbin/pbs_mom: Unable to configure temporary directory.
pbs_mom startup failed, exit 1 aborting.

When executing

sudo /etc/init.d/pbs restart

as stated in section 15.8.2 of the Admin Guide.

Could someone please help?

Thanks a lot.

Alfonso

2

Please try to set this on /scratch/tmp and restart pbs services
chmod 1777 -R /scratch/tmp

3

Thanks for your answer adarsh. Same error message. Those are the attributes of /scratch/tmp.
rwxrwxrwt 3 root root 4096 sep 14 14:50 tmp

Error message is again:
pbs_mom: Operation not permitted (1) in tmp_file_sec, Security violation “/scratch/tmp” resolves to “/scratch”
/opt/pbs/sbin/pbs_mom: Unable to configure temporary directory.
pbs_mom startup failed, exit 1 aborting.

4

Could you please try this

mkdir -p /myscratch/tmp
chmod 1777 -R /myscratch/tmp

set this as PBS_TMPDIR in the /etc/pbs.conf , restart the pbs services on the compute node.
If it works, then there must be some sticky bit set for /scratch/tmp (from the RAID or mount )
namei -mo /scratch
namei -mo /scratch/tmp

5

From the error message:

This is trying to tell us something. Why does the MoM think /scratch/tmp is the same as /scratch?

To gather more information, you should run these on the MoM host:

ls -lid /scratch /scratch/tmp /scratch/ /scratch/tmp/
df /scratch /scratch/tmp
grep -w scratch /proc/mounts

Another thing to check: You say /scratch/tmp comes from a different drive. You need to make sure the local mount point is mode 0755. That is, unmount /scratch/tmp and /scratch then check the modes on each directory:

umount /scratch/tmp    # Might get "not mounted" error
umount /scratch        # ditto
ls -ld /scratch /scratch/tmp
1 Like