Max files too low

Users/Site Administrators
1

Hi
Problem:

Max files too low - you may want to increase it

How to increase it?

I read this information but it tells me nothing:

Max files allowed = %ld
Logs the nfiles currently configured.
Log level: PBSEVENT_DEBUG | PBSEVENT_DEBUG2
Max files too low - you may want to increase it
If nfiles is <1024, the pbs_comm daemon emits the message. If nfiles configured is <100, the startup aborts.
Usually nfiles must be configured to allow the number of connections (usually the number of MoMs) the
pbs_comm process is going to handle.

2

Anyone can remark in this topic?

3

Hi, I am interested in an answer as well, please explain what is this configuration effects, how to change it and when.

Thanks,
Roy

4

u can increase the nfiles limit using ulimit command before starting pbs_comm, or u can make a permanent change using systemctl… see this page for help: How to Increase Number of Open Files Limit in Linux

1 Like
5

Thanks @subhasisb , can you please explain what are the instructions as to decide which value should we set it to?

Is it based on number of Moms? Number of jobs? etc.

Thanks,
Roy

6

pbs_comm is a broker - it is the component that mediates messages between the server/mom and between moms. It consumes one file descriptor per tcp connection. These endpoints (server, moms) that connect to the same pbs_comm consumes one fd, each. So, the number of moms primarily dictate the value of nfiles u need for comm.

Of course, some fds are already consumed by the pbs_comm process in libraries, configuration files etc (we can keep aside a hundred for that). Thus, if you have 2000 moms, then you might want to set nfiles to something like 2500…

7

Thank you @subhasisb thats very helpful information

8

I also encountered the problem of “Max files allowed = 1024, Max files too low - you may want to increase it”
All of the following operations are performed by root

My steps are as follows:
(1)The default ulimit -n of my OS is 1024.
So when the PBS service is started, PBS_mom, pbs_sched,pbs_serveres all prompt as follows (no pbs_comm):

			01/07/2022 18:01:16;0d80;pbs_mom;TPP;pbs_mom(Main Thread);Max files allowed = 1024
			01/07/2022 18:01:16;0c06;pbs_mom;TPP;pbs_mom(Main Thread);Max files too low - you may want to increase it.
			01/07/2022 18:01:36;0d80;pbs_sched;TPP;pbs_sched(Main Thread);Max files allowed = 1024
			01/07/2022 18:01:36;0c06;pbs_sched;TPP;pbs_sched(Main Thread);Max files too low - you may want to increase it.
			01/07/2022 18:02:02;0d80;Server@cf-poc-master;TPP;Server@cf-poc-master(Main Thread);Max files allowed = 1024
			01/07/2022 18:02:02;0c06;Server@cf-poc-master;TPP;Server@cf-poc-master(Main Thread);Max files too low - you may want to increase it.

(2)Then I change ulimit -n 102400 in /etc/security/limits.conf:
root soft nofile 102400
root hard nofile 102400

(3)Then restart the machine and confirm that the modification takes effect
command “ulimit -n” output 102400

But, when PBS service starts again, pbs_mom, pbs_sched and pbs_server(except pbs_comm) still report warning mean that 1024 too low like above,
then I try command “ulimit -n 102400” in the current login shell, it still no effect

Why?

Should I modify ulimit in the pbs’s systemd service conf file,or by systemctl, or there maybe some conf in pbs overwrite the OS config?

Looking forward to your reply, thank you!