We have met a PBS pro system limit for stack size in our HPC running. We found that all the jobs were submitted by PBS pro have a stack size limit to 16384.
Is there a configuration file in PBS pro so that we can remove this restriction and get unlimited stack size.
Even we changed in our .bashrc file to ulimit -s unlimited, and also set up unlimited stack size in /etc/security/limits.conf, but it is still the same for 16384 limit
Is there any solution in PBS Pro to set up to ulimit -s unlimited?
Thanks Jon, it is working great. I also set up the stack size to unlimited in /etc/security/limits.conf so that it won’t have any hard/soft limit in the system level.
/etc/security/limits.conf only applies to processes whose ancestor has called the limits PAM module, and PBSPro currently does not use PAM modules.
In other words, except for the limits explicitly set by PBSPro MoM itself from the job resource requests (e.g. through pvmem or pmem), the job processes inherit MoM’s own limits (unless of course you e.g. change them in an /etc/profile.d script to something else if a task starts out as a login shell).
By the way, setting the stacksize limit to “unlimited” can be dangerous for threaded programs in Linux: anything but the master pthread needs to preallocate virtual address space for the stack, and obviously “unlimited” won’t fly for that, and you often end up with less stack space (typically just 2MB) for the extra pthreads, unless the program takes care to actually explicitly ask for more before pthreads are created.
Better set it to something close to the physical memory of the machine instead (if you have 64 bit machines, there’s plenty of address space to burn) if you suspect you might have to run threaded programs with multiple threads needing quite some stack space.