Openpbs kubernates integration

Users/Site Administrators
1

Hi ,

I am using openpbs 22.05 and wanted to integrate Kubernetes installation has been done but the after creating pod job is not showing via qstat command below I shared relevant details

Create hooks and set their properties.

Create and define hook pbs-kubernetes

create hook pbs-kubernetes
set hook pbs-kubernetes type = site
set hook pbs-kubernetes enabled = true
set hook pbs-kubernetes event = execjob_end
set hook pbs-kubernetes event += execjob_launch
set hook pbs-kubernetes user = pbsadmin
set hook pbs-kubernetes alarm = 30
set hook pbs-kubernetes order = 1
set hook pbs-kubernetes debug = false
set hook pbs-kubernetes fail_action = none
import hook pbs-kubernetes application/x-python base64 -

import hook pbs-kubernetes application/x-config base64 -

.yaml file for creating pod

apiVersion: v1
kind: Pod
metadata:
name: ubuntu3
spec:
containers:

  • name: ubuntu
    image: ubuntu:latest
    command: [“sleep”, “3600”]
    resources:
    requests:
    memory: “640Mi”
    cpu: “1”
    limits:
    memory: “700Mi”
    cpu: “3”

[root@master ~]# qstat --version
pbs_version = 22.05.11

I was following below link

GitHub - saimanikant/kubernetes-pbspro-connector: Kubernetes connector for pbs pro

[root@master ~]# kubectl version
Client Version: v1.30.14
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.0

I would appreciate if any tips.

Thanks

Abhishek

2

Please check

  • the PBS Mom logs on the compute node. Check whether it has any messages from the hook pbs-kubernetes
  • the custom scheduler logs
3

Hi Adarsh,

mom log has below content when I submit the job using qsub
02/02/2026 12:42:07;0002;pbs_mom;Svr;pbs_mom;ipv4 interface cni0: kubecompute.example.com
02/02/2026 12:42:07;0002;pbs_mom;Svr;pbs_mom;ipv6 interface lo: localhost6.localdomain6
02/02/2026 12:42:07;0002;pbs_mom;Svr;pbs_mom;ipv6 interface ens18: kubecompute.example.com
02/02/2026 12:42:07;0002;pbs_mom;Svr;pbs_mom;ipv6 interface flannel.1: kubecompute.example.com
02/02/2026 12:42:07;0002;pbs_mom;Svr;pbs_mom;ipv6 interface cni0: kubecompute.example.com
02/02/2026 12:42:07;0002;pbs_mom;Svr;pbs_mom;ipv6 interface veth1f876f77: kubecompute.example.com
02/02/2026 12:42:07;0100;pbs_mom;Req;;Type 1 request received from root@192.168.0.95:15001, sock=0
02/02/2026 12:42:09;0006;pbs_mom;Hook;pbs_python;K8S PROLOGUE ERROR for job 94.master.example.com: Command ‘[’/usr/bin/sudo’, ‘-u’, ‘pbsuser’, ‘/usr/bin/kubectl’, ‘apply’, ‘-f’, ‘/tmp/pbs-job-94-master-example-com.yaml’, ‘–kubeconfig’, ‘/opt/pbs/kubeconfigs/pbs-runner.conf’]’ returned non-zero exit status 1.
02/02/2026 12:42:09;0006;pbs_python;Hook;pbs_python;PROLOGUE HOOK FIRED for job 94.master.example.com on host kubecompute
02/02/2026 12:42:09;0008;pbs_mom;Job;94.master.example.com;Started, pid = 3047618

which is correct way to submit job either kubectl apply -f redis.yaml or using qsub.

Is their any to check all my configuration are correct or not?

Thanks

Abhishek

4

Sometimes its exit with 175 after submitting the pod

2026/02/02 13:19:49
Starting Scheduler Iteration
2026/02/02 13:20:09 End of Iteration
2026/02/02 13:20:09
Starting Scheduler Iteration
2026/02/02 13:20:12 exit status 175

Thanks

Abhishek

5

This is the correct way to submit workload to the kubernetes cluster, here the intent is to run this workload in a pod that runs on the compute node attached PBS Pro cluster.

qsub is way to submit HPC jobs to the PBS cluster
kubeclt apply -f redis.yaml is way to submit job to kubernet cluster
The compute node – can be part of the PBS cluster running hpc workload (submitted via qsub) or the compute node can become worker node (running the workload submitted via kubectl -f redis.yaml redis.yaml)