# Design a Hook to monitor the Resources utilized by a Job **URL:** \ **Category:** Users/Site Administrators\ **Created:** [January 7, 2019, 7:31pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384 "2019-01-07T19:31:35Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![rgarg](https://avatars.discourse-cdn.com/v4/letter/r/5f8ce5/32.png) [@rgarg](https://community.openpbs.org/u/rgarg)\ **Post date:** [January 7, 2019, 7:31pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/1 "2019-01-07T19:31:35Z")
Hi, Qstat -fx gives us the status of each job which includes the number of resources used and requested. Objective: When a job finishes its execution I need to place the below contents in a file and send a comparison to the user. resources\_used.mem = 4700kb resources\_used.ncpus = 1 resources\_used.walltime = 00:49:100 Resource\_List.mem = 1048576kb Resource\_List.ncpus = 1 Resource\_List.walltime = 72:00:00 Could this be implemented using a Hook? If yes how? As per my understanding I believe we have to create a Hook using “execjob\_end”. In documentation there isn’t any example which uses “execjob\_end”. It would be great if any relevant example could be shared. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [January 7, 2019, 10:02pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/2 "2019-01-07T22:02:31Z")
This can be achieved in the **execjob\_epilogue** hook 1. check if the node is mother superior, if it is 2. write to a file or write to stdout 3. Please try and let us know Sample code: > ``` > import pbs,os,sys > e=pbs.event() > j=e.job > jobid=job.id > path_to_stdout_file=j._stdout_file > pbs.logmsg(pbs.LOG_DEBUG,"The path of stdout file is >>>>> %s" % (path_to_stdout_file)) > if (j.in_ms_mom()): > fileopen=(path_to_stdout_file,'a') > fileopen.write(str(j.Resource_List['ncpus'])) > fileopen.close() > > ``` You can used the below values to populate the file j.resources\_used.ncpus, j.Resource\_List.mem, j.resources\_used.mem , etc . Please test/share and let us know ---
**Author:** ![agurban](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/agurban/32/67_2.png) [@agurban](https://community.openpbs.org/u/agurban)\ **Post date:** [January 7, 2019, 10:55pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/3 "2019-01-07T22:55:01Z")
Accidental reply. Re-sent to correct recipient. ---
**Author:** ![rgarg](https://avatars.discourse-cdn.com/v4/letter/r/5f8ce5/32.png) [@rgarg](https://community.openpbs.org/u/rgarg)\ **Post date:** [January 9, 2019, 3:33pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/4 "2019-01-09T15:33:13Z")
Hi Adarsh, Thanks for the sample code. I will try it and share the results. ---
**Author:** ![rgarg](https://avatars.discourse-cdn.com/v4/letter/r/5f8ce5/32.png) [@rgarg](https://community.openpbs.org/u/rgarg)\ **Post date:** [January 9, 2019, 6:58pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/5 "2019-01-09T18:58:06Z")
Hi Adarsh, Seems I am making some error while defining #PBS -W stageout in job script. Could you please share the code for just writing to a file. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [January 11, 2019, 9:26am UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/6 "2019-01-11T09:26:42Z")
Hi Rakhen, #PBS -W stagein = \@:\ #PBS -W stageout = \@:\ execution\_path: filename in jobs’s staging & execution directory on execution host storage\_path: filename on host hostname The “ **@** ” character separates execute path specification from storage path specification (it is a separator) example: qsub -Wstagein=.@storageservername:/root/dummy/_.txt -Wstageout=_.fem@storageservername:/root/temp1 – /bin/sleep 5 Thank you ---
**Author:** ![rgarg](https://avatars.discourse-cdn.com/v4/letter/r/5f8ce5/32.png) [@rgarg](https://community.openpbs.org/u/rgarg)\ **Post date:** [January 14, 2019, 6:52pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/7 "2019-01-14T18:52:24Z")
Hi Adarsh, Please refer to the below code: import pbs import sys import os try: e=pbs.event() pbs.logmsg(pbs.LOG\_DEBUG, “executed epilogue hook”) if e.job.in\_ms\_mom(): f=open(“abc.txt”, “w+”) f.write(str(e.job.Resource\_List[“ncpus”])) f.write("\n") f.write(str(e.job.Resource\_List[“walltime”])) f.write("\n") f.write(str(e.job.Resource\_List[“place”])) f.write("\n") f.close() else: f=open(“abc.txt”, “w+”) f.write(“This is line1”) f.close() except SystemExit: pass except: e.reject(“Failed to write to a file”) The code is working fine for accessing Resource\_List objects. When I am trying to access resources\_used objects the job is getting aborted. ![image](https://canada1.discourse-cdn.com/flex030/uploads/pbspro/original/1X/2b96c9979487b1b1d1b4915ebad7b72229f92487.png) ![image](https://canada1.discourse-cdn.com/flex030/uploads/pbspro/original/1X/860f5e6d188e241a740f30532a4e755885c13ca7.png) I had even tried to access resources\_used objects by implementing the below example but still the job end up with abort status… ![image](https://canada1.discourse-cdn.com/flex030/uploads/pbspro/original/1X/b281379edad8a16d4a71435be94d28e7b9842842.png) Note: I have not implemented stdout concept. ---
**Author:** ![adarsh](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@adarsh](https://community.openpbs.org/u/adarsh)\ **Post date:** [January 14, 2019, 8:45pm UTC](https://community.openpbs.org/t/design-a-hook-to-monitor-the-resources-utilized-by-a-job/1384/8 "2019-01-14T20:45:42Z")
Please try the below as execjob\_epilogue hook: f=open(’/tmp/abc.txt’, ‘w+’) f.write("======================================================================================\n\n\t\t\tResource Usage on %s:\n\nJobId: %s \n\tJob\_Name = %s \n\tJob\_Owner = %s \n\tresource\_used.cpupercent = %s \n\tresource\_used.cput = %s \n\tresources\_used.mem = %s \n\tresources\_used.ncpus = %s \n\tresources\_used.vmem = %s \n\tresources\_used.walltime = %s \n\tqueue = %s \n\tserver = %s \n\tAccount\_Name = %s \n\tError\_Path = %s \n\texec\_host = %s \n\texec\_vnode = %s \n\tJoin\_Path = %s \n\tKeep\_Files = %s \n\tOutput\_Path = %s \n\tRerunable = %s \n\tResource\_List.ncpus = %s \n\tResource\_List.nodect = %s \n\tResource\_List.place = %s \n\tResource\_List.mem = %s \n\tResource\_List.cput = %s\n\tResource\_List.walltime = %s \n\tjobdir = %s \n\tEnvironment\_Variables = PBS\_O\_SYSTEM=%s,PBS\_O\_SHELL=%s,PBS\_O\_HOME=%s,PBS\_O\_LOGNAME=%s,PBS\_O\_WORKDIR=%s,PBS\_O\_LANG=%s,PBS\_O\_PATH=%s,PBS\_O\_QUEUE=%s,PBS\_O\_HOST=%s \n\teuser = %s \n\tegroup = %s \n\trun\_count = %s \n\t\n======================================================================================\n" % (str(datetime.datetime.now()),ji,j.Job\_Name,j.Job\_Owner,j.resources\_used.cpupercent,j.resources\_used.cput,j.resources\_used.mem,j.resources\_used.ncpus,j.resources\_used.vmem,j.resources\_used.walltime,j.queue,j.server,j.Account\_Name,j.Error\_Path,j.exec\_host2,j.exec\_vnode,j.Join\_Path,j.Keep\_Files,j.Output\_Path,j.Rerunable,j.Resource\_List[‘ncpus’],j.Resource\_List[‘nodect’],j.Resource\_List[‘place’],j.Resource\_List[‘mem’],j.Resource\_List[‘cput’],j.Resource\_List[‘walltime’],j.jobdir,j.Variable\_List[“PBS\_O\_SYSTEM”],j.Variable\_List[“PBS\_O\_SHELL”],j.Variable\_List[“PBS\_O\_HOME”],j.Variable\_List[“PBS\_O\_LOGNAME”],j.Variable\_List[“PBS\_O\_WORKDIR”],j.Variable\_List[“PBS\_O\_LANG”],j.Variable\_List[“PBS\_O\_PATH”],j.Variable\_List[“PBS\_O\_QUEUE”],j.Variable\_List[“PBS\_O\_HOST”],j.euser,j.egroup,j.run\_count)) f.close() If you improvise it, please share it with the community