Failed to start PBS dataservice-lock seems to be held

I am getting failed to acquire lock problem.
Someone in the forum had a similar problem and they had corrected it.
The suggested solutions are:- 1.To clean /var folder if it is full(My /var folder is only 74% full)
2.Few lines be added in pbspro/sbin/pbs_dataservice to check the ownership of /var/run/postgresql.
if it is different from the database user defined in /var/spool/pbspro/server_priv/db_user, the ownership should be changed accordingly.(I don’t have postgresql directory created in /var/run)
Here is what I am getting-
[root@host1 datastore]# service pbs restart
Restarting PBS
Stopping PBS
PBS mom - was pid: 117648
PBS sched - was pid: 117662
PBS comm - was pid: 117614
Waiting for shutdown to complete
Starting PBS
PBS comm
/opt/pbs/sbin/pbs_comm ready (pid=119554),Threads:4
PBS mom
PBS sched
Connecting to PBS dataservice…Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1 cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1 - cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1- cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 119947 running on host: host1 - cannot start]
continuing in background.
PBS server
Please help me.

Looks like another instance of the database may be running. What is PID 119947?

The following command will show you processes related to PBS…
ps -ef | grep -i pbs

Please make sure all PBS Pro processes are stopped before you try to start the service.

I stopped all pbs processes and then restarted.Everytime some other process with different pid seems to hold the lock.I have the same problem
[root@host1 kbharadw]# sudo /etc/init.d/pbs stop
Stopping PBS
PBS mom - was pid: 1072123
PBS sched - was pid: 1072137
PBS comm - was pid: 1072085
Waiting for shutdown to complete
[root@host1 datastore]# ps -ef | grep -i pbs
root 1073924 3148393 0 10:13 pts/1 00:00:00 grep --color=auto -i pbs
[root@host1 datastore]# sudo /etc/init.d/pbs start
Starting PBS
PBS comm
/opt/pbs/sbin/pbs_comm ready (pid=1074020), Threads:4
PBS mom
PBS sched
Connecting to PBS dataservice…Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1- cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1 - cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1 - cannot start]
.Failed to start PBS dataservice:[Failed to acquire lock on /var/spool/pbs/datastore/pbs_dblock. Lock seems to be held by pid: 1074411 running on host: host1 - cannot start]
continuing in background.
PBS server
[root@host1 datastore]#

Please run the following command:

sudo /opt/pbs/sbin/pbs_ds_monitor check

Run it once with PBS running and once with PBS stopped. Please capture the return code (echo $?) and any output produced.

did you try rm /tmp/.s.postgresql.15007*.

if theses file exists, you need to delete it manually as well as pbs_dblock file in $PBS_HOME/databases

Also make sure the permission of /tmp is 1777
hope it helps

1 Like

Thank you!
It worked by deleting the files and changing the owner of “/tmp” to root and permissions to 777.

The permissions of tmp should be 1777, rather than 0777. The “sticky bit” should be set on temporary directories.

1 Like