I have a question about how environment variables are set when running pbs jobs with openPBS. On one of my clusters it seems that my $PATH and $LD_LIBRARY_PATH are set correctly on the nodes, but are not available when I run a job with PBS.
On this cluster, my .pbs file is able to run at the command line after sshing to a compute node, but it will not run when I submit with qsub from the head node because the job cannot find the correct library. Upon further examination, I found that the $PATH and $LD_LIBRARY_PATH variables have different values when I echo them from a compute node terminal vs. outputting the values of those variables from a running pbs job.
My question is: how does openPBS set the $PATH and $LD_LIBRARY_PATH variables for pbs jobs? I understand that I can set these values manually within my .pbs file or pass them with ‘qsub -v’ but I would like to understand where to configure the default values or where PBS pulls them from (mostly for standardization across my clusters). Did I miss something during openPBS configuration to specify these?