Periodically monitor actual CPU & Memory used by each job

Users/Site Administrators
1

I am looking for a lightweight way to periodically (e.g. every 10 minutes) monitor the actual CPU & Memory of every job.
The goal would be to inject the data into Elasticsearch.

Are there any open source solutions for it?

Thanks,
Roy

2

You can try to write a json parser that would parse the qstat -fx -F json output every 10 minutes and find out the jobid/ resources_used.mem / resources_used.vmem / resources_used.cput for each of the jobs and you can use data collectors/injectors to lodge this information in Elasticsearch.

3

Thanks adarsh, this might work.