# Comm daemon and failover **URL:** \ **Category:** Users/Site Administrators\ **Created:** [July 22, 2021, 10:12am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709 "2021-07-22T10:12:48Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 22, 2021, 10:12am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/1 "2021-07-22T10:12:48Z")
Hi, Do we need set PBS\_COMM\_ROUTERS parameter in pbs.conf on secondary host in failover configuration? ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 22, 2021, 10:43am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/2 "2021-07-22T10:43:02Z")
No, in case of failover PBS\_COMM\_ROUTERS does not need to be set Regards, Subhasis ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 22, 2021, 11:36am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/3 "2021-07-22T11:36:22Z")
Thanks for reply. OK, so it is normal that comm daemon do not start on secondary server when primary is running and it is active: ```auto [root@secondary ~]# /etc/init.d/pbs start Starting PBS PBS comm pbs_comm: another PBS comm router running at the same port PBS server [root@secondary ~]# ``` ??? ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 23, 2021, 5:16am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/4 "2021-07-23T05:16:28Z")
Ah no, that error you are seeing means something is occupying the same port on the machine “Secondary”. netstat -pant | grep 17001 should show you what process is using that port. ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 28, 2021, 11:34am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/5 "2021-07-28T11:34:17Z")
Hmm… “netstat -pant | grep 17001” display nothing: ```auto [user@secondary ~]$ sudo ps -ax | grep pbs 327647 pts/0 S+ 0:00 grep --color=auto pbs [user@secondary ~]$ sudo netstat -pant | grep 17001 [user@secondary ~]$ sudo /etc/init.d/pbs start Starting PBS PBS comm PBS server [user@secondary ~]$ pbs_comm: another PBS comm router running at the same port [user@secondary ~]$ sudo netstat -pant | grep 17001 [user@secondary ~]$ sudo ps -ax | grep pbs 327729 ? Ss 0:00 /opt/pbs/sbin/pbs_server.bin 327736 pts/0 S+ 0:00 grep --color=auto pbs [user@secondary ~]$ ``` How to troubleshooting this? > **secondary pbs.conf** > > ```auto > PBS_PRIMARY=pri > PBS_SECONDARY=sec > PBS_SERVER=primary > PBS_START_SERVER=1 > PBS_START_SCHED=0 > PBS_START_COMM=1 > PBS_START_MOM=0 > PBS_EXEC=/opt/pbs > PBS_HOME=/shared/location > PBS_CORE_LIMIT=unlimited > PBS_SCP=/usr/bin/scp > PBS_RCP=/usr/bin/false > PBS_LEAF_NAME=sec > PBS_COMM_LOG_EVENTS=2047 > > ``` > **primary pbs.conf** > > ```auto > PBS_PRIMARY=pri > PBS_SECONDARY=sec > PBS_SERVER=primary > PBS_START_SERVER=1 > PBS_START_SCHED=1 > PBS_START_COMM=1 > PBS_START_MOM=0 > PBS_EXEC=/opt/pbs > PBS_HOME=/shared/location > PBS_CORE_LIMIT=unlimited > PBS_SCP=/usr/bin/scp > PBS_RCP=/usr/bin/false > PBS_LEAF_NAME=pri > > ``` > **gdb output** > > ```auto > (gdb) break lock_out > Breakpoint 1 at 0x4045c0: lock_out. (2 locations) > (gdb) run > Starting program: /opt/pbs/sbin/pbs_comm > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib64/libthread_db.so.1". > > Breakpoint 1, lock_out (fds=6, op=2) at pbs_comm.c:228 > 228 (void) lseek(fds, (off_t) 0, SEEK_SET); > (gdb) n > 229 flock.l_type = op; > (gdb) n > 231 flock.l_start = 0; > (gdb) n > 232 flock.l_len = 0; > (gdb) n > 234 if (fcntl(fds, F_SETLK, &flock) != -1) { > (gdb) n > > Breakpoint 1, lock_out (op=2, fds=6) at pbs_comm.c:235 > 235 if (op == F_WRLCK) { > (gdb) n > go_to_background () at pbs_comm.c:293 > 293 rc = fork(); > (gdb) n > [Detaching after fork from child process 327935] > 294 if (rc == -1) /* fork failed */ > (gdb) pbs_comm: another PBS comm router running at the same port > n > 296 if (rc > 0) > (gdb) n > 297 exit(0); /* parent goes away, allowing booting to continue */ > (gdb) n > [Inferior 1 (process 327920) exited normally] > > ``` > **comm log** > > ```auto > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Log;Log opened > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;pbs_version=20.0.0 > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;pbs_build=mach=N/A:security=N/A:configure_args=N/A > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;hostname=secondary.domain;pbs_leaf_name=sec;pbs_mom_node_name=N/A > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;ipv4 interface lo: localhost4.localdomain4 > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;ipv4 interface bond0.1: secondary.domain > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;ipv4 interface bond0.2: sec > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;ipv6 interface lo: localhost6.localdomain6 > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;ipv6 interface bond0.1: secondary.domain > 07/28/2021 14:19:49;0002;Comm@sec;Svr;Comm@sec;ipv6 interface bond0.2: secondary.domain > 07/28/2021 14:19:49;0d80;Comm@sec;TPP;Comm@sec(Main Thread);TPP authentication method = resvport > > ``` > **server logs related to secondary host** > > ```auto > 07/28/2021 14:36:39;0100;Server@primary;Req;;Type 27 request received from root@secondary.domain, sock=24 > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;hostname=secondary.domain;pbs_leaf_name=sec;pbs_mom_node_name=N/A > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;ipv4 interface lo: localhost4.localdomain4 > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;ipv4 interface bond0.1: secondary.domain > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;ipv4 interface bond0.2: sec > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;ipv6 interface lo: localhost6.localdomain6 > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;ipv6 interface bond0.1: secondary.domain > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;ipv6 interface bond0.2: secondary.domain > 07/28/2021 14:36:38;0006;Server@secondary;Fil;Server@secondary;Version 20.0.0, started, initialization type = 1 > 07/28/2021 14:36:38;0002;Server@secondary;Svr;Server@secondary;pbs_server: coming up as Secondary, Primary is pri > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@ > > ``` I saw in server log something like this: @^@^@^@^@^ What does that mean? ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 28, 2021, 2:15pm UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/6 "2021-07-28T14:15:54Z")
gdb output with breakpoint set on go\_to\_background function: ```auto (gdb) run Starting program: /opt/pbs/sbin/pbs_comm [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Breakpoint 1, go_to_background () at pbs_comm.c:514 514 go_to_background(); (gdb) n 293 rc = fork(); (gdb) n [Detaching after fork from child process 328580] 294 if (rc == -1) /* fork failed */ (gdb) pbs_comm: another PBS comm router running at the same port p rc $6 = 328580 (gdb) n 296 if (rc > 0) (gdb) n 297 exit(0); /* parent goes away, allowing booting to continue */ (gdb) n [Inferior 1 (process 328579) exited normally] (gdb) n The program is not being run. ``` > **code of go\_to\_background function** > > ```auto > /** > * @brief > * Forks a background process and continues on that, while > * exiting the foreground process. It also sets the child process to > * become the session leader. This function is avaible only on Non-Windows > * platforms and in non-debug mode. > * > * @return - pid_t - sid of the child process (result of setsid) > * @retval >0 - sid of the child process. > * @retval -1 - Fork or setsid failed. > */ > static pid_t > go_to_background() > { > pid_t sid = -1; > int rc; > > lock_out(lockfds, F_UNLCK); > rc = fork(); > if (rc == -1) /* fork failed */ > return ((pid_t) -1); > if (rc > 0) > exit(0); /* parent goes away, allowing booting to continue */ > > lock_out(lockfds, F_WRLCK); > if ((sid = setsid()) == -1) { > fprintf(stderr, "pbs_comm: setsid failed"); > return ((pid_t) -1); > } > already_forked = 1; > return sid; > } > > ``` I think that problem is witch go\_to\_background function in src/server/pbs\_comm.c file. When it forking comm process, it has been running two comm deamon processes at the same time and on the same port. Right? Problem maybe is more complex. When go\_to\_background function create child process of comm deamon, it goes through lock\_out function which return error: “another PBS comm router running at the same port”. So lock\_out function is problematic in this case. On the other hand, why this function work well on primary server??? When I kill comm on primary, comm on secondary starts well (by hand of course). When I run qterm -s on primary(serwer and scheduler was shutdown), on secondary scheduler and datastore starts automatically (server was start prior), but comm daemon do not stop on primary. So I run on primary: kill -TERM \, then it stops on primary but not start on secondary. When I now starts pbs\_comm on secondary “by hand”, it’s start well. qstat -Bf shows that server\_host is primary.domain - but primary is shutdown and secondary takes control. **Is this normal behaviour?** ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 29, 2021, 10:45am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/7 "2021-07-29T10:45:58Z")
The problem is not in the go\_to\_background() function, it is also not a bug. This is a problem with your hostname resolution setup. The actual secondary hostname’s FQDN should match the FQDN of the name listed in PBS\_SECONDARY. In your case, the server\_host of the secondary is set via PBS\_LEAF\_NAME as sec. This is not matching with the FQDN of the secondary machine, which is probably [sec.org](http://sec.org). So, kindly change this: Change: PBS\_LEAF\_NAME=sec.fqdn and then it should work. ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 29, 2021, 10:47am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/8 "2021-07-29T10:47:08Z")
If this works, kindly log this as an issue, we will update the code to make it slightly more robust! ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 29, 2021, 11:19am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/9 "2021-07-29T11:19:23Z")
I am not sure that I understand well what I need to do. I need change PBS\_LEAF\_NAME variable to FQDN hostname or short hostname with dot ? If it should be a FQDN hostname, I need change /etc/hosts configuration to something like this: 192.169.0.1 primary.domain 192.169.0.2 secondary.domain 192.168.0.1 primary pri 192.168.0.2 secondary sec where 192.169.x.x is external network and 192.168.x.x is internal network that I want to pbs communicate in ??? Now I have configured /etc/hosts like this: 192.168.0.1 primary.domain primary pri 192.168.0.2 secondary.domain secondary sec ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 29, 2021, 11:31am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/10 "2021-07-29T11:31:37Z")
Given the above /etc/hosts, try setting: PBS\_LEAF\_NAME=secondary.domain ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 29, 2021, 11:41am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/11 "2021-07-29T11:41:37Z")
> [@subhasisb](#): > > Given the above /etc/hosts, Which? With 192.169.x.x or without? ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 29, 2021, 11:44am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/12 "2021-07-29T11:44:59Z")
Without the 192.169.x.x network. So you need this: > [@boboshaq](#): > > 192.168.0.1 primary.domain primary pri > 192.168.0.2 secondary.domain secondary sec and this: > [@subhasisb](#): > > PBS\_LEAF\_NAME=secondary.domain This way, the value in PBS\_LEAF\_NAME matches the FQDN of the secondary host, secondary.domain exactly ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 29, 2021, 12:08pm UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/13 "2021-07-29T12:08:14Z")
Yes, now it is working! Comm daemons starts on primary and secondary without error. Comm daemon registered his leafs only on internal network that is correct. When I terminate primary, secondary take control(starts dataservice and scheduler, comm and server starts prior). I ask in this post how to set PBS\_LEAF\_NAME and conclusion was that I need use short hostname - not with domain. So conclusions from that post are wrong: > [@PBS\_PRIMARY/PBS\_SECONDARY vs PBS\_LEAF\_NAME](https://community.openpbs.org/t/pbs-primary-pbs-secondary-vs-pbs-leaf-name/2651): > > Hi, Can You tell me that I understanding well meaning PBS\_RIMARY and PBS\_SECONDARY parameter when I have been set PBS\_LEAF\_NAME parameter according which this table: parameters in pbs.conf on server1(primary) on server2(secondary) PBS\_SERVER h1.domain h2.domain PBS\_LEAF\_NAME comm\_h1 comm\_h2 PBS\_PRIMARY comm\_h1 comm\_h1 PBS\_SECONDARY comm\_h2 comm\_h2 And I need add to file /etc/hosts alias: comm\_h1 on server1 and comm\_h2 to the same file on server2, yes? Regards… SOLUTION and conclusion is: **PBS\_LEAF\_NAME should be FQDN of hostname which resolves to IP of internal network!** Many thanks for Your help! Regards! ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 30, 2021, 9:22am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/14 "2021-07-30T09:22:00Z")
@subhasisb I want to ask, diffidently, do You have an idea, why server\_host variable after running command: qstat -Bf, when secondary takes control, is still setted as “primary.domain” hostname? This is not a big problem to me now, but I am curious, do I have something bad in my configuration again? ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 30, 2021, 9:58am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/15 "2021-07-30T09:58:53Z")
Hi you can look at the PBS admin guide for the right steps to configure this. I think it is still the case of configuration issue, so yes, this is not correct (the server\_host should show the secondary host-fqdn when secondary is active). From the admin guide: PBS\_PRIMARY FQDN of hostname Hostname of primary server host. If you set PBS\_LEAF\_NAME on the primary server host, make sure that PBS\_PRIMARY matches PBS\_LEAF\_NAME on the corre- sponding host. If you do not set PBS\_LEAF\_NAME on the server host, make sure that PBS\_PRIMARY matches the hostname of the server host. PBS\_SECONDARY FQDN of hostname Hostname of secondary server host. If you set PBS\_LEAF\_NAME on the secondary server host, make sure that PBS\_SECONDARY matches PBS\_LEAF\_NAME on the corresponding host. If you do not set PBS\_LEAF\_NAME on the server host, make sure that PBS\_SECONDARY matches the host- name of the server host. PBS\_SERVER Hostname Name of primary server host. Cannot be longer than 255 characters. If the short name of the server host resolves to the correct IP address, you can use the short name for the value of the PBS\_SERVER entry in pbs.conf. If only the FQDN of the server host resolves to the correct IP address, you must use the FQDN for the value of PBS\_SERVER. Please check with settings based on this. ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [July 30, 2021, 12:39pm UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/16 "2021-07-30T12:39:19Z")
Hmm… Everything looks OK: | | primary host | secondary host | compute nodexx | | --- | --- | --- | --- | | PBS\_PRIMARY | primary.domain | primary.domain | primary.domain | | PBS\_SECONDARY | secondary.domain | secondary.domain | secondary.domain | | PBS\_SERVER | primary | primary | primary | | PBS\_LEAF\_NAME | primary.domain | secondary.domain | nodexx.domain | I also checked it with FQDN of PBS\_SERVER but result was without change. It is not so important, I think. So I will not be longer take care of this topic. Regards and thanks for advice! ---
**Author:** ![subhasisb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/subhasisb/32/12_2.png) [@subhasisb](https://community.openpbs.org/u/subhasisb)\ **Post date:** [July 30, 2021, 3:33pm UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/17 "2021-07-30T15:33:34Z")
Hmm that is strange. Please attach the pbs.conf from primary and secondary and your /etc/hosts from both the machines. Regards, Subhasis ---
**Author:** ![boboshaq](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@boboshaq](https://community.openpbs.org/u/boboshaq)\ **Post date:** [August 20, 2021, 8:48am UTC](https://community.openpbs.org/t/comm-daemon-and-failover/2709/18 "2021-08-20T08:48:05Z")
> **pbs.conf primary** > > ```auto > PBS_PRIMARY=primary.domain > PBS_SECONDARY=primary.domain > PBS_SERVER=primary > PBS_LEAF_NAME=primary.domain > PBS_START_SERVER=1 > PBS_START_SCHED=1 > PBS_START_MOM=0 > PBS_START_COMM=1 > PBS_EXEC=/opt/pbs > PBS_HOME=/path/to/shared/pbs_home > PBS_CORE_LIMIT=unlimited > PBS_SCP=/bin/scp > PBS_RCP=/usr/bin/false > > ``` > **pbs.conf secondary** > > ```auto > PBS_PRIMARY=primary.domain > PBS_SECONDARY=secondary.domain > PBS_SERVER=primary > PBS_LEAF_NAME=secondary.domain > PBS_START_SERVER=1 > PBS_START_SCHED=0 > PBS_START_COMM=1 > PBS_START_MOM=0 > PBS_EXEC=/opt/pbs > PBS_HOME=/path/to/shared/pbs_home > PBS_CORE_LIMIT=unlimited > PBS_SCP=/bin/scp > PBS_RCP=/usr/bin/false > PBS_COMM_LOG_EVENTS=511 > > ``` > **hosts primary** > > ```auto > 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 > ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 > 192.168.0.1 primary.domain primary pri > 192.168.0.2 secondary.domain secondary sec > 192.168.0.10 node01.domain node01 n1 > > ``` > **hosts secondary** > > ```auto > 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 > ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 > 192.168.0.1 primary.domain primary pri > 192.168.0.2 secondary.domain secondary sec > 192.168.0.10 node01.domain node01 n1 > > ```