So I’ve discovered that Hooks can be used to run a script at the end of a job…but not in the way that I’d need.
I am obviously missing something here.
I want to go through the accounting logs at the end of a job and get the resources_used details from it in order to let users know how efficient they were.
The only hooks that execute on the completion of a job all run on the hosts the job ran on? And it looks like these jobs are already sending the data back for the accounting logs. But there doesn’t seem to be an option to deliver that information back to the user in a usable format in the case of an array job?
I’ve been told that I can use an event such as
such as at “execjob_end: Event After Job Cleanup” event. The hook
executes on all the hosts allocated to a job, after a job is cleaned
up. It can read attribute array_indices_remaining - which should help
with reporting the statistics after the whole array job finishes, and
not after each subjob.
By my reading, it looks like the way to do this is to wait until the job finishes with the array_indices_remaining == 0, then to trigger a script on the head node to scrape and analyse the accounting logs?
Alternatively I send the information at the end of each job directly to a db somewhere, wait until the job finishes with the array_indices_remaining == 0, then to trigger a script on the head node to send a summary from the db?