I’d remove interface 9. Accounting logs are there for two reasons. First is a trace of states a job/resv went through from its inception to when it leaves the system. The other is for actual accounting (e.g., PBSA). Knowing how many seconds the user was willing to wait doesn’t fall into either one.
Hmm, now that I’m thinking about this more, if we try and compare this to a job, the records that are printed are for actual state changes. The job start. The job was requeued. There isn’t one that says we tried to requeue the job and then one afterwards that said we were successful. Do we want the N and z records? Think about this from the point of view of what someone would want to know after the fact. Do they want to know that an alter request came in and was rejected? I’m not sure. That sounds more like a debug type message. There are server log messages for that. I think people will only want to know what has really happened to their reservations, not things that might have happened. Even if a reservation alter request was accepted, do they care that there is a record that says one was attempted? I think they will only care that the reservation was altered. I don’t think we need to proliferate the size of our accounting log unnecessarily.
I think the only record we need out of the new proposed records is the ‘Z’ record. Speaking of the ‘Z’ record, what happens on a normal reservation reconfirmation (state degraded to confirmed). Should we just reuse that record? We are reconfirming the reservation.
I do feel that we need these accounting logs as they have information on the timings that were requested and later confirmed/denied. There are also state changes involved from CO/RN --> AL --> CO/RN.
Prakash, I see the utility of a stable interface to extract the information covered by the accounting log messages in interface 9, but that doesn’t seem within the remit of the accounting log. MIght it not be better in the daemon log?
You don’t mention anything in the EDD about searching for a new set of vnodes in the case where the reservation start or end request can’t be completed on the existing nodes. Shouldn’t that be something you talk about?
As for the accounting records:
first a typo in interface 9
Details - When a reservation alter is being requested in non-interactive mode, a new type record…
should be:
Details - When a reservation alter is being requested in interactive mode, a new type record…
as for new records or not, we already have a precedent of including records to track timing of various events. For example:
U - Unconfirmed reservation created on server.
Y - Reservation confirmed by the scheduler.
B - Reservation record, written at the beginning of a reservation period.
or
P - Provisioning starts for job or reservation
p - Provisioning ends for job or reservation.
or to some extent
Q - Job entered a queue
S - Job execution started.
Since Analytics only looks at the accounting logs I’d think it would be useful to have the information as described in the EDD. I might eliminate the “z” record and just use the “Z” record to say request was denied in place of the requested changes, the requested change information is already captured in the “N” record.
I’m still not convinced that we need all of the new records. PBSA only looks at the E record. They get all the information they want from it. The rest of the records aren’t useful to them. The simulator uses the ‘E’ and the ‘B’ records. This means they only care about what actually happened, not what might have happened.
As for ‘p’ and ‘P’, I think the ‘p’ record was a mistake. There should be a server log record that provisioning started, but it doesn’t belong in the accounting log.
While the reservation is changing state, we don’t log all of the job state changes (W->Q, R->S, etc) in the accounting log. I’m not sure we need to care about this one.
We should definitely have a reservation reconfirmation record. I actually think we should reuse the ‘Y’ record and enhance it to fit our purposes. After an alter, we are reconfirming the reservation. It’s really just like a reconfirmation after the reservation went into degraded state. The only difference is that the start/end times may have changed.
I think all of this data should be logged, just not in the accounting log. I think it’s better placed in the server log. This was Ian’s point. I agree with him.
I agree with @bhroam here. I’m not sure use case are we trying to address by logging when ralter request was made/denied (in interactive/non-interactive mode). All that matters is that whether it was reconfirmed.
We don’t log all job state changes to accounting record then why take a different path for reservations.
I also feel that reservation alter request record and reservation confirmation record should be logged as they have information on the timings that were requested and later confirmed/denied.
To assure the timing of state changes involved from CO/RN --> AL --> CO/RN.
The log we want in accounting log or in server log that could be decide .
Also I have a small doubt on
B. Requirements:
4) It shall be possible to move forward the end time of a reservation so that it ends sooner.
Can we rephrase this line ?
I understood from “move forward” that we are expecting to change the end time to future time(towards the future), so how it will reflect the reservation end sooner ?I s my understanding about above line is correct ?
Do this have any impact on simulation. I know that we have an issue with standing reservations since we don’t provided the necessary information in the accounting logs. If we can properly simulate the changes by reading information in the accounting logs then I think a log in the daemon is fine. However, if we don’t have enough information to simulate these changes then I recommend we add a new record to the accounting logs. Adding in @agrawalravi90 to see what he thinks.
Thank you for all the inputs and time for having a discussion over the call. I have updated the Interface document based on the conclusion of removing the ‘N’, ‘Z’ and ‘z’ record and improving the ‘Y’ record.
Request everyone to review the latest version and provide feedback.
I am of the view that this is a requirement and will not lead to any interface change, that’s why there is no mention of it in the Interface Design. Please let me know if you feel otherwise.
Thanks Prakash. I’ve only looked at Interface 8 (Y record), and feel that we could enhance it a bit more.
I’d imagine that, for standing reservations, one will be able to alter the recurrence rule and timezone attributes as well. If that’s true, then I think it’s very important to capture those two attributes as well in the Y record. If that is not true, even then it’d be a good idea to capture them as that information is not captured in the accounting logs anywhere (https://pbspro.atlassian.net/browse/PP-703), but it’s upto you.
Also, I am not sure that logging the index of a particular occurrence (bullet ‘d’ under ‘Details’) is going to be very valuable inside a Y record. The B records logs index information for particular occurrences anyways, and I feel that’s the correct place for it as we see a B record for every occurrence.
We will not be able to change the recurrence rule and the timezone through pbs_ralter. However, thinking about it, I do see there is a possibility of having the user being allowed to change the recurrence rule, but for changing timezone of a reservation, I am not able to come up with a use case for that.
Thinking further, I do not see any use of recurrence rule being part of the accounting logs. How will it help?
As we are using the ‘Y’ record for a reconfirmation, we need to know which instance onwards a standing reservation will be affected by the requested and (now confirmed) change. That is what I am thinking, please let me know your thoughts.
I previously mentioned that a degraded reservation gets a ‘Y’ record when it is reconfirmed. I was incorrect. I looked through the code and tested it and no record is printed when a degraded reservation is reconfirmed. I guess it goes with the fact that no record is printed when a reservation goes into the degraded state.
I am happy with what @prakashcv13 has proposed. The only addition I could see is maybe the nodes. Since the nodes can change with the ralter, it might be nice to know that it happened.
This could be useful to the simulator. The simulator can do a better job simulating if it knows that the nodes of a reservation changed in the middle of its confirmed life.
Well, recurrence rule & timezone are the essence of a standing reservation. One of the use cases where they might help a lot is debugging. If there was a bug with a standing reservation which is not in the system anymore, there’d be no way to know whether the different occurrences, as logged inside accounting, actually happened at the correct times or not, without recurrence rule & timezone information. So, as I said, it’s a nice to have if this RFE will not allow modifying them. If it will allow modifying them, then I think it’s very important that we log them.
I’m ok with logging the index, I see now how it can be useful. Thanks for clarifying.
I do agree with Bhroam that logging the exec_vnode after ralter can be useful information.
Prakash, the nodes information that will be logged for standing reservations, will it be the complete list (for all occurrences), or just the next occurrence? I think you might wanna clarify that in the EDD.