PP-261: micro-second time stamp for daemon logging

Developers
1

Hi All,

I have posted an EDD for micro-second logging for all PBS daemons in the following link. Please review it and provide your comments.

https://pbspro.atlassian.net/wiki/display/PD/PP-261%3A+micro-second+time+stamp+for+daemon+logging

2

When PBS Pro is configured to log to syslog, how does this new feature impact syslog messages?

3

A couple of things

What about windows? I don’t think you can query microseconds in windows.
Do we want this enabled by default? Microsecond logging is useful for benchmarking or performance testing. Is it useful in the normal case or will it just add extra length to our log lines.
The direction we are going with our configuration options is to add them to qmgr. I don’t know if pbs.conf is the right place for this. I understand it is the most convenient.

If we stick with pbs.conf, could you change the interface title from “switch” to "pbs.conf option"
Could you either extend the synopsis or remove it. Just printing the option name isn’t useful.

4

Hi Bhroam,

As you said we don’t have a direct way to query microseconds in Windows but it can be achieved by writing a wrapper for gettimeofday using the APIs like QueryPerformanceFrequency and QueryPerformanceCounter. We need to try this and find the accuracy.

Enabling microsecond logging by default has come as requirement.

As you suggested we are planning to move this configuration to qmgr.

5

As per our knowledge this feature does not impact syslog.

6

I think you should explicitly state that this feature does not apply when PBS Pro is configured to log via syslog.

1 Like
7

I just had a conversation with Carl about turning this feature on by default. He reminded me that the log message format is a stable interface. We can’t change the default without giving notice of at least one year. I suggest you go back to who added your requirement that this feature is turned on by default and have them rethink their decision.

Bhroam

8

That sounds fine to me to have it off by default.

9

All,

I have updated the EDD and incorporated the comments received. Please look into and give your further comments/sign off.

Thanks,
Suresh

10

Thanks for making the changes Suresh

It looks fine to me.

Bhroam

11

EDD looks fine to me.

12

@suresht

One minor comment… please also specify that
qmgr -c "set server highres_time_log_enable=0"
also disables high-res logging. The same as unsetting the value (default is off).

I assume this is what the implementation would have done anyway.

13

@mkaro
Thanks for your comments. I have updated the EDD accordingly.

14

I’m not sure why we are setting this in qmgr for all daemons. Shouldn’t this be set on the local node level for mom and comms (in pbs.conf until we have a better place to handle this for the daemons) since they can be run on different machines and should not have to contact the server to determine which level of time stamp they should be using? This seems fine to me for the server and scheduler since the scheduler can run if it can talk to the server. However, the comm and mom can run without being able to contact the server.

15

From an end-user point of view, it seems much better to present a single interface for choosing which timestamp to log. It would be inconvenient to set multiple interfaces to change what ought to be uniform across the product.

16

Fully agree. However, until we move all of the daemon configurations to qmgr then I don’t want to do piece mail.

17

I disagree that we should wait until we move everything to qmgr to start moving in the right direction. If we continue on the path we are going now, we’ll just have that many more options to worry about being backwards compatible with.

Bhroam

2 Likes
18

I think this statement by Subhasis applies here as well. Until we decide to start/staff “all configurations via qmgr project” we should continue to use the pbs.conf file for configuring daemons that do not currently get their configurations from qmgr.

19

If the idea is to enable microsecond logging on a per-node basis, then I agree that pbs.conf is the right place for this configuration data. If it is a global option, then I believe that qmgr is the right place. The design document does not specify whether it is per-node or global. Please update the design document.

20

@mkaro
I have updated the design document in order to say that it is node level option and hence we are going with pbs.conf. Please look into it and provide your sign off.