Local scratch Directory use

Users/Site Administrators
1

Dear Team,

I need help.
How to use the local scratch directory of compute nodes from the PBS scripts.

In setup One master node and 6 compute nodes.

Master node does not have the scratch directory but in all compute nodes having local scratch directory. Any configuration is required in /var/spool/pbs/mom_priv/config file.

Please help on this.

2

For example:

  1. cn1 to cn6 are the compute nodes, on each of the node
    mkdir /scratch/cn$i
    chmod 1777 -R /scratch

  2. on each of the node
    source /etc/pbs.conf
    cd $PBS_HOME/mom_priv
    edit config and add the below line
    $jobdir_root /scratch/cn$i # if it is cn1 , then it would be $jobdir_root /scratch/cn1
    restart the pbs services on the compute node

  3. Now submit a job as below
    qsub -W sandbox=PRIVATE – /bin/sleep 1000
    qstat -f | grep jobdir

For example if the job is running on cn1 : then you would see
/scratch/cn1/pbs..< servename >.x8z

Please refer this documentation: https://help.altair.com/2024.1.0/PBS%20Professional/PBS2024.1.pdf
Section:
10.13 Job Directories
10.13.1.6 Staging and Execution Directory Caveats
10.13.1.1 Using Job-specific Staging and Execution Directories
Search word: jobdir_root

3

Thanks for the reply.

My requirement to use the local scratch directory for the temp files.
How to use the local scratch directory for the temporary files.

4

default location to store temporary files is : /var/tmp

Please search this document for $tmpdir and PBS_TMPDIR

edit $PBS_HOM/mom_priv/config
$tmpdir /scratch/cn1

  1. PBS_TMPDIR enviroment variable

  2. you popuplate /etc/pbs.conf with PBS_TMPDIR=/scratch/cn1

Restart of the pbs services would be required to take this variable into account, if any configuration files are updated.

5

Thank you for reply.

Can we set the user based . When users submit jobs user the users directory of local scratch for temp file for example /scratch/user.
please guide how to do and could you please share the script and configurations.
It will help me.

6

No, the above is not user based.
The user can use tmpdir locaiton if the application batch command line argument supports it.
Otherwise, it is setup per the configuration for the system.

If you could share the script or explain the workflow (what you would like to achieve) that would be helpful and the community members can suggest.

7

If it helps

qsub -v MYTEMPDIR=/scratch/user – /bin/sleep 100
or

in a script
#PBS -v MYTEMPDIR=/scratch/user
#PBS -N jobname

/bin/sleep 100
env