# PBS MoM Log Analysis in ELK Stack
**URL:** https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637
**Category:** Users/Site Administrators
**Created:** [June 7, 2019, 7:38pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637 "2019-06-07T19:38:09Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@rgarg](https://community.openpbs.org/u/rgarg)
#### Post date: [June 7, 2019, 7:38pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/1 "2019-06-07T19:38:09Z")
Hi,
I am looking forward to visualize mom\_logs on Kibana in Elk Stack. As a prerequisite we need to design a Grok Filter. As a part of the setup I do have setup for ELK Stack ready. I am able to send PBS mom\_logs to kibana as well.
Example of Log and its filter in logstash
Log: localhost GET /v2/applink/5c2f4bb3e9fda1234edc64d 400 46ms 5bc6e716b5d6cb35fc9687c0
Grok Filter: %{WORD:environment} %{WORD:method} %{URIPATH:url} %{NUMBER:response\_status} %{WORD:response\_time} %{USERNAME:user\_id}
For e.g as this mom log has multiple factors like date, time stamp, job id. My query is how do I get to know about the parameters that are to be defined in the Grok Filter
Log: 06/04/2019 18:12:05;0008;pbs\_mom;Job;224.primaryserver1;Started, pid = 2352
Thanks,
Rakhen
---
### Author:  [@adarsh](https://community.openpbs.org/u/adarsh)
#### Post date: [June 7, 2019, 9:04pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/2 "2019-06-07T21:04:17Z")
The log record format is the same as used by other PBS daemons, with the addition of the thread number and the daemon name in the log record. The log record format is as follows:
`date-time;event_code;daemon_name(thread number);object_type;object_name;message`
**event\_code :** Please refer the Table 2-38: tracejob Filters in the RG-229 ( [https://www.pbsworks.com/pdfs/PBSReferenceGuide19.2.1.pdf](https://www.pbsworks.com/pdfs/PBSReferenceGuide19.2.1.pdf))
**daemon\_name:** pbs\_mom
**object\_type :** type of object the message is about Job, Que, Svr, Req, Node,Hook
**object\_name:** name of the specific object
**message\_text :** text of the log message
It is good to see your project with mom\_logs and Kibana, please share your experiences on this with the community once you have some results. This might spawn some interests and discussions.
---
### Author:  [@rgarg](https://community.openpbs.org/u/rgarg)
#### Post date: [June 10, 2019, 2:01pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/3 "2019-06-10T14:01:05Z")
Thanks Adarsh for the details. Sure I will share my experience.
---
### Author:  [@rgarg](https://community.openpbs.org/u/rgarg)
#### Post date: [June 17, 2019, 7:55pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/4 "2019-06-17T19:55:14Z")
Hi All,
**Project: PBS MoM log Analysis in ELK Stack**
Below are my leanings from the project:
1. Just to give an overview ELK Stack consists of four main components i.e: Elasticsearch, Logstash, Kibana and Filebeat(to be installed on client machine).
2. Elasticsearch, Logstash, Kibana were installed on Centos7 instance and Filebeat was installed on PBS Pro Cluster(Has shared file system).
3. Filebeat can be installed on multiple clients in-case the log data is coming from different sources.
4. Grok filter in logstash helps us reading pbs\_mom logs. It needs to be designed as per your requirements.
5. Analyzed PBS mom\_logs through Kibana and have created a sample dashboard which shows visualizations of the data. Below are the snapshots:
Any further ideas are highly welcome!!
Thanks,
Rakhen
---
### Author:  [@adarsh](https://community.openpbs.org/u/adarsh)
#### Post date: [June 18, 2019, 7:26am UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/5 "2019-06-18T07:26:51Z")
Excellent ! . Thank you Rakhen. It is indeed very useful. The same can be extended to
1. Server logs : To trace job acceptance throughput, common error charting
2. Scheduler logs : say number of jobs considered at each scheduling cycle, common error charting
3. Comm logs : mom registration
If you have the plan to share it, then push it to github and so that community can use, contribute and extend.
4. Accounting logs : $PBS\_HOME/server\_priv/accounting
Well done !
---
### Author:  [@rgarg](https://community.openpbs.org/u/rgarg)
#### Post date: [June 18, 2019, 2:04pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/6 "2019-06-18T14:04:12Z")
Thank you so much Adarsh…
---
### Author:  [@subhasisb](https://community.openpbs.org/u/subhasisb)
#### Post date: [June 27, 2019, 4:51am UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/7 "2019-06-27T04:51:37Z")
Looks awesome @rgarg
Is it possible to get some walk through document on how to get all of these configured. I had been meaning to play with either the ELK or EFK stack for a long time now, but never found the time.
Thanks!
---
### Author:  [@rgarg](https://community.openpbs.org/u/rgarg)
#### Post date: [June 27, 2019, 7:15pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/8 "2019-06-27T19:15:49Z")
Hi Adarsh,
Please refer to the below link:
> **[Install Elasticsearch, Logstash, and Kibana (ELK Stack) on CentOS 7](https://www.itzgeek.com/how-tos/linux/centos-how-tos/updated-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7-rhel-7.html)**
>
> ELK stack is also known as Elastic stack, consists of Elasticsearch, Logstash, and Kibana; helps you to have all of your logs stored in one
Thanks,
Rakhen
---
### Author:  [@datakid](https://community.openpbs.org/u/datakid)
#### Post date: [May 4, 2021, 1:18am UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/9 "2021-05-04T01:18:22Z")
> [@subhasisb](#):
>
> Is it possible to get some walk through document on how to get all of these configured. I had been meaning to play with either the ELK or EFK stack for a long time now, but never found the time.
Likewise, I’d love to see your logstash conf for `mom_logs`. There’s a [git repo](https://community.openpbs.org/t/pbs-accounting-and-metrics-and-elastic-stack/816/5) mentioned that I’ll take a look at too.
---
### Author:  [@subhasisb](https://community.openpbs.org/u/subhasisb)
#### Post date: [May 6, 2021, 4:27am UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/10 "2021-05-06T04:27:08Z")
Just FYI - I recently did a grafana visualization of the PBS logs based on loki and promtail and it was very straightforward.
---
### Author:  [@ravikiran.s](https://community.openpbs.org/u/ravikiran.s)
#### Post date: [January 23, 2022, 2:43pm UTC](https://community.openpbs.org/t/pbs-mom-log-analysis-in-elk-stack/1637/11 "2022-01-23T14:43:44Z")
Hi subhasisb,
Can You Please share Entire Configuration Step.It will be helpfull.
Thanks