PP-662, PP-663: UCR and External Interface document for Reservation enhancements

[prakashcv13]

@bhroam,

My personal opinion is to allow the shortened instance to run, as with that we will give a chance to the admin/user to extend it if needed. If we simply skip it, we are not giving any chance for an extension.

Thanks,
Prakash

[agrawalravi90]

Thanks for clarifying Prakash. So, the instances won’t really get skipped, they will just run for the duration of time that’s left for them.

But, instead of handling the conflicted cases and running the reservations partially, possibly leading to issues which Bhroam highlighted, why don’t we just reject an alter request if it conflicts with another instance of that reservation? Won’t that be way more intuitive and less confusing ?

[prakashcv13]

@bhroam

First, the user submits jobs that are the full length of the occurrences. We are just setting them up to fail. We will start a job that will be killed at the end of the occurrence.

Well, the users do have the option of extending the next instance as well.

@bhroam

Second, a long occurrence immediately followed by a shorted occurrence looks like one really long occurrence. The problem is that we kill all running jobs between the two occurrences. There are reasons to do this (e.g., the two occurrences are on different sets of nodes), but I’m not sure users will understand why their jobs were killed while their reservation is still running.

Why wouldn’t they? It is the admin or themselves who will be altering the reservation.

@bhroam

So is this case like when the server comes up in the middle of an occurrence?

No it is not. But this is something the user will know beforehand, which is not always true for server going down and coming back up.

Thanks,
Prakash

[prakashcv13]

@agrawalravi90

why don’t we just reject an alter request if it conflicts with another instance of that reservation? Won’t that be way more intuitive and less confusing ?

I would leave this to be decided by @smgoosen and the PM team.
My take is that there can be other events between the end time of the next instance and the start time of the subsequent instance which conflict with the alter. So far, it is also not guaranteed that all the instances of a standing reservation will run on the same set of nodes. So, if the user is altering a reservation so that it conflicts with the next instance, this is what he/she intends to do and would know the consequence. If I am the user to me looking for another set of nodes and reduced time of the subsequent instance is more intuitive.

[bhroam]

@prakashcv13

Well, the users do have the option of extending the next instance as well.

They do, but they have to do it after this occurrence has finished. You only have the ability to alter the next occurrence. This means they’ll need to wait until the current occurrence finishes, extend the next one, wait until that occurrence finishes, extend the next one, etc. This is not a user friendly interface.

@prakashcv13

Why wouldn’t they? It is the admin or themselves who will be altering the reservation.

You bring up a good point of who is going to use this command. If the admin extends the reservation, the user could be clueless that this happened. They could be confused. If they do the alter themselves, then they’ll need to be a savvy user to understand exactly what it means to alter a standing reservation. Of course you probably need to be a savvy user to use pbs_ralter in the first place.

@prakashcv13

My take is that there can be other events between the end time of the next instance and the start time of the subsequent instance which conflict with the alter. So far, it is also not guaranteed that all the instances of a standing reservation will run on the same set of nodes. So, if the user is altering a reservation so that it conflicts with the next instance, this is what he/she intends to do and would know the consequence. If I am the user to me looking for another set of nodes and reduced time of the subsequent instance is more intuitive.

You are not covering my use case. I think it is a real use case too. However intuitive it is that you get a long and a short occurrence, the user might only submit jobs that are near the length of the reservation. One of the main use cases of this RFE is to allow the user to extend a reservation when they know their jobs aren’t going to finish in time. This could force them to extend the next, and the next, etc.

There is one other way we could solve this use case. It would be a change to how standing reservations work. We could just not start jobs that wouldn’t end before the end of the occurrence. The functionality of running all jobs in a reservation regardless of if they would finish in time was meant for advance reservations. Since the whole reservation is going away after its time is up, there is no reason to not start all the jobs that we can. There is always a possibility it might finish early. This changes with standing reservations. Should we start a job that we know can’t finish in time but could in a future occurrence?

@prakashcv13

I would leave this to be decided by @smgoosen and the PM team.

My take is that there can be other events between the end time of the next instance and the start time of the subsequent instance which conflict with the alter. So far, it is also not guaranteed that all the instances of a standing reservation will run on the same set of nodes. So, if the user is altering a reservation so that it conflicts with the next instance, this is what he/she intends to do and would know the consequence. If I am the user to me looking for another set of nodes and reduced time of the subsequent instance is more intuitive.

You are correct that not all occurrences of a standing reservation will be on the same nodes. This doesn’t cover Ravi’s point though. He’s talking about conflicting in time, not in space. In this case, is a shortened occurrence the most intuitive? Would it be more intuitive to start both? They don’t conflict on the nodes they are running on. There’s no real reason the second occurrence couldn’t start on time. Now this doesn’t fit with our standing reservation framework. This would cause huge problems for us. I’m not suggesting we do it. I’m just bringing it up as an alternative to what the user might think.

I read through the use cases and acceptance criteria of PP-662 and it doesn’t talk about this. You are right, the PMs will have to extend the use case to include this.

[prakashcv13]

Hi All,

Me, @bhroam and @smgoosen had a discussion and decided that if the requested times cause an overlap/shadowing of a later instance of the standing reservation being modified, we would deny the alter request, until we have a customer that thinks otherwise.

I have updated the UCR and the design to reflect this.

Request everyone to review.

Thanks,
Prakash

[bhroam]

Hey @prakashcv13
It looks like Interface 1.4.i.iii .4 conflicts with Interface 4.4. The first says either that if the ralter is confirmed, you’ll get CONFIRMED back. You’ll otherwise get UNCONFIRMED. In 4.4 says if the ralter is denied, DENIED comes back. I assume 4.4 is correct.

I’d flip Interface 7 around. I’d say “If the current occurrence of a standing reservation being altered interferes with a later occurrence, …”

A suggestion for the new message: “Requested time will interfere with a later occurrence of the standing reservation” or something like that. I’d definitely use the term occurrence since that is what we call each instance of a standing reservation.

I would collapse interface 14 4.a and 4.b into one bullet. You basically say the same thing for both of them. Why not just say “for advance and standing reservations” (also, currently 14.4.b says when a standing reservation is confirmed, the server logs says denied).

Interface 15: use occurrence instead of instance.

[prakashcv13]

Hi @bhroam,

@bhroam

It looks like Interface 1.4.i.iii .4 conflicts with Interface 4.4. The first says either that if the ralter is confirmed, you’ll get CONFIRMED back. You’ll otherwise get UNCONFIRMED. In 4.4 says if the ralter is denied, DENIED comes back. I assume 4.4 is correct.

The corresponding server log for 1.4.iiii.4 is Interface 3. Now I have made some changes to 1.4.i.iii. These changes will clarify the CLI better.

@bhroam

I’d flip Interface 7 around. I’d say “If the current occurrence of a standing reservation being altered interferes with a later occurrence, …”

For a standing reservation, it is not always the current occurrence, it can be the next one as well, this is what we have followed throughout the document. But, I have updated the statement to make it simpler to read.

@bhroam

A suggestion for the new message: “Requested time will interfere with a later occurrence of the standing reservation” or something like that. I’d definitely use the term occurrence since that is what we call each instance of a standing reservation.

Changed.

@bhroam

I would collapse interface 14 4.a and 4.b into one bullet. You basically say the same thing for both of them. Why not just say “for advance and standing reservations” (also, currently 14.4.b says when a standing reservation is confirmed, the server logs says denied).

14.4.b is corrected - I like them to be separate bullet points.

@bhroam

Interface 15: use occurrence instead of instance.

Done.

[bhroam]

Hey @prakashcv13
Thanks for making the updates. It’s much more clear now.

I’m happy with it. I sign off.

Bhroam

[varunsonkar]

Hi @prakashcv13,
In the Design document I see the mention of CLI message in case when alter request in interactive mode is denied, as:
"pbs_ralter: DENIED"
But I do not see the CLI message for the same in the non interactive mode.
Will it be same or different?

[prakashcv13]

Hi @varunsonkar,

Please see Interface 5.

Thanks,
Prakash

[smgoosen]

I sign off on the addition of the -q option to interface 1