PBS accounting and metrics and Elastic stack

Users/Site Administrators
1

Hi everyone,

Just to share with the community (I don’t know really how yet), at the HPC cluster in CNES, we have plugged some information coming from PBS into an Elastic stack instance:

  • PBS Sever accounting logs: thanks to this, we are able to view some statistics about our users job and cpu hours consumption. I give only a quick glance at PBS analytics, but this seems to be quite the same goal, but with more history capacity and reactivity, and also some real time information.
  • qstat and pbsnodes insights: Indexing every five minutes the information from these commands, we are able to have a detailed view of cluster occupation and user behavior.

So as mentionned, I don’t really know how to share this right now, but if somebody is interested, we can think about it :slight_smile: !

Here are some screenshot of created Dashboard (sorry, legends in french):
Qstat:

image
image.jpg1705×826 182 KB

As a new user I can only put one image…

2 Likes
2

Thanks for sharing – looks interesting.

3

Another snapshot based on accounting analytics:

image
image.jpg1631×761 163 KB

4

And Qstat by user here:

image
image.jpg1631×591 187 KB

5

@guillaumeeb : This is interesting to me and the community members might also be interested. If it is not classified and/or private, then you could host it on github or its alternatives.

6

As this was asked to me a few times, and it was on my todo list for more than a year, here are the most important config files for the acounting part:

2 Likes
7

I forgot to mention, I tried to explain a bit, but if you have more question, do not hesitate!

2 Likes
8

Thanks for this @guillaumeeb, it’s very useful.

9

Glad it’s still useful for someone :slight_smile:

I responded to your github issue too.