# PBS accounting and metrics and Elastic stack

**URL:** <https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816>\
**Category:** Users/Site Administrators\
**Created:** [January 9, 2018, 9:25pm UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816 "2018-01-09T21:25:20Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![guillaumeeb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/guillaumeeb/32/104_2.png) [@guillaumeeb](https://community.openpbs.org/u/guillaumeeb)\
**Post date:** [January 9, 2018, 9:25pm UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/1 "2018-01-09T21:25:21Z")

</div>

Hi everyone,

Just to share with the community (I don’t know really how yet), at the HPC cluster in CNES, we have plugged some information coming from PBS into an Elastic stack instance:

- PBS Sever accounting logs: thanks to this, we are able to view some statistics about our users job and cpu hours consumption. I give only a quick glance at PBS analytics, but this seems to be quite the same goal, but with more history capacity and reactivity, and also some real time information.
- qstat and pbsnodes insights: Indexing every five minutes the information from these commands, we are able to have a detailed view of cluster occupation and user behavior.

So as mentionned, I don’t really know how to share this right now, but if somebody is interested, we can think about it 🙂 !

Here are some screenshot of created Dashboard (sorry, legends in french):  
Qstat:

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/pbspro/original/1X/b3107776567ef3922e86bf4a53647ff16e421258.jpg)

As a new user I can only put one image…

---

<div class="post-metadata">

**Author:** ![billnitzberg](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/billnitzberg/32/6_2.png) [@billnitzberg](https://community.openpbs.org/u/billnitzberg)\
**Post date:** [January 10, 2018, 4:15am UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/2 "2018-01-10T04:15:59Z")

</div>

Thanks for sharing – looks interesting.

---

<div class="post-metadata">

**Author:** ![guillaumeeb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/guillaumeeb/32/104_2.png) [@guillaumeeb](https://community.openpbs.org/u/guillaumeeb)\
**Post date:** [January 10, 2018, 10:10am UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/3 "2018-01-10T10:10:30Z")

</div>

Another snapshot based on accounting analytics:

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/pbspro/original/1X/d3d2ab477eceb06fd41c35187838c930fb3855fa.jpg)

---

<div class="post-metadata">

**Author:** ![guillaumeeb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/guillaumeeb/32/104_2.png) [@guillaumeeb](https://community.openpbs.org/u/guillaumeeb)\
**Post date:** [January 10, 2018, 10:16am UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/4 "2018-01-10T10:16:58Z")

</div>

And Qstat by user here:

 ![image](https://canada1.discourse-cdn.com/flex030/uploads/pbspro/original/1X/a47d8bfdbce1d3b5f7245768dafad949c3af5a01.jpg)

---

<div class="post-metadata">

**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\
**Post date:** [September 20, 2019, 1:13pm UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/5 "2019-09-20T13:13:57Z")

</div>

@guillaumeeb : This is interesting to me and the community members might also be interested. If it is not classified and/or private, then you could host it on github or its alternatives.

---

<div class="post-metadata">

**Author:** ![guillaumeeb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/guillaumeeb/32/104_2.png) [@guillaumeeb](https://community.openpbs.org/u/guillaumeeb)\
**Post date:** [September 20, 2019, 7:28pm UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/6 "2019-09-20T19:28:13Z")

</div>

As this was asked to me a few times, and it was on my todo list for more than a year, here are the most important config files for the acounting part:

> **[guillaumeeb/pbs-elastic](https://github.com/guillaumeeb/pbs-elastic)**
>
> Some script and configurations files to index a PBS cluster information into Elastic stack. - guillaumeeb/pbs-elastic

---

<div class="post-metadata">

**Author:** ![guillaumeeb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/guillaumeeb/32/104_2.png) [@guillaumeeb](https://community.openpbs.org/u/guillaumeeb)\
**Post date:** [September 20, 2019, 7:30pm UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/7 "2019-09-20T19:30:08Z")

</div>

I forgot to mention, I tried to explain a bit, but if you have more question, do not hesitate!

---

<div class="post-metadata">

**Author:** ![datakid](https://avatars.discourse-cdn.com/v4/letter/d/db5fbb/32.png) [@datakid](https://community.openpbs.org/u/datakid)\
**Post date:** [May 4, 2021, 1:29am UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/8 "2021-05-04T01:29:18Z")

</div>

Thanks for this @guillaumeeb, it’s very useful.

---

<div class="post-metadata">

**Author:** ![guillaumeeb](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/guillaumeeb/32/104_2.png) [@guillaumeeb](https://community.openpbs.org/u/guillaumeeb)\
**Post date:** [May 28, 2021, 2:52pm UTC](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/9 "2021-05-28T14:52:27Z")

</div>

Glad it’s still useful for someone 🙂

I responded to your github issue too.
