# PP-482: Non-destructive walltime **URL:** \ **Category:** Developers\ **Created:** [June 24, 2017, 12:23am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587 "2017-06-24T00:23:23Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\ **Post date:** [July 24, 2017, 6:21pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/21 "2017-07-24T18:21:57Z")
I’d prefer to leave soft\_walltime as visible to all users. I don’t see harm in it being exposed to users in something like qstat -f when it is labeled as exactly what it is, I just didn’t think it should be shown in the alternate qstat output when users will not actually request it themselves and it might cause confusion. As for estimated.soft\_walltime, either a job attribute or a log message used in testing constitutes a public interface and you need this interface regardless of known customer driven need. Since we have to have one or the other and they are both on equal footing I’d opt for the one that is cleaner and more useful, and that is the job attribute in my opinion. Thanks! ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [September 1, 2017, 11:10pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/22 "2017-09-01T23:10:43Z")
I’ve updated the EDD slightly. I added estimated.start\_time back in for automated testing purposes. I realized that I need to be able to test if the estimates are being correctly increased. The only way to do this would be a log message that is printed per-job per-cycle even on cycles where when the soft\_walltime didn’t increase, or an attribute. I decided to go the attribute route. Bhroam ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [September 5, 2017, 11:54pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/23 "2017-09-05T23:54:57Z")
Another slight update to the EDD: I’ve added the error message which is printed when the soft\_walltime \> walltime ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [October 6, 2017, 7:55pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/24 "2017-10-06T19:55:03Z")
Good EDD! Zooming in on the following: "If both a soft\_walltime and a hard walltime are set, the soft\_walltime will never be extended past the job’s hard walltime. Job K has a soft\_walltime=1:00:00 and a hard walltime=1:30:00 When K exceeds its soft\_walltime, it would have been extended to 2:00:00. Since 2:00:00 is past its hard walltime, K is extended to 1:30:00 instead." [Al] Going further, what happens when Job k has soft\_walltime=1:30:00 and job K exceeds this limit also. Will you print a log message about not being able to extend anymore? ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [October 6, 2017, 8:24pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/25 "2017-10-06T20:24:24Z")
> [@bayucan](#): > > [Al] Going further, what happens when Job k has soft\_walltime=1:30:00 and job K exceeds this limit also. Will you print a log message about not being able to extend anymore? I don’t believe a log message is required. If K exceeds 1:30:00, it will have exceeded its hard walltime and will be killed. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [October 6, 2017, 8:36pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/26 "2017-10-06T20:36:15Z")
True, thanks for the clarification. ---
**Author:** ![jon](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/jon/32/23_2.png) [@jon](https://community.openpbs.org/u/jon)\ **Post date:** [October 11, 2017, 2:05am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/27 "2017-10-11T02:05:31Z")
Changes to the EDD look good to me. I am excited to see this get added to the product. One question, you define set\_soft\_walltime as type=long. Does PBS care that a long is defined as a duration or is that standard behavior? ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [October 11, 2017, 6:17pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/28 "2017-10-11T18:17:28Z")
> [@jon](#): > > One question, you define set\_soft\_walltime as type=long. Does PBS care that a long is defined as a duration or is that standard behavior? The underlying type of walltime (and soft\_walltime) is a long. There is just some extra glue on top of it that allows you to set HH:MM:SS to the resource. This is where pbs.duration() comes into play. Bhroam ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [October 16, 2017, 11:56pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/29 "2017-10-16T23:56:05Z")
I realized today that the EDD was not complete. It did not talk about the interaction between soft\_walltime and preempt\_order. If preempt\_order is based on the percentage the job is complete, the soft\_walltime is used to calculate this percentage. An example is during the first extension, the job will go from 99% to 50% complete. This can change how a job is preempted. Please take another look at the EDD and tell me if it’s OK. Bhroam ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [October 17, 2017, 12:47am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/30 "2017-10-17T00:47:40Z")
Bill disliked the name non-destructive walltime. He thought it was worse than Soft Walltime. I changed it to Soft Walltime (I can’t change the name of this thread). ---
**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\ **Post date:** [October 17, 2017, 4:44am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/31 "2017-10-17T04:44:15Z")
This seems reasonable to me, thanks for catching this and updating the EDD. ---
**Author:** ![jon](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/jon/32/23_2.png) [@jon](https://community.openpbs.org/u/jon)\ **Post date:** [October 18, 2017, 3:57pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/32 "2017-10-18T15:57:36Z")
Thanks for catching this. Personally, I think it would be better to have the scheduler not reduce the percentage calculation back down to 50% if we extend the walltime due to a job running longer than initially predicted. Instead, I think it makes sense to just leave the job at 100% so that it will not be preempted. The reasoning for this is that ideally when the admin predicts the walltime they should be close in most cases to the actual time. If not, then it would be anticipated that the job is almost done. If we preempt it then we are most likely preempting a job that is almost done. I would rather that we preempt jobs that are not as far along as on that we had to extend the walltime on. ---
**Author:** ![anamika](https://avatars.discourse-cdn.com/v4/letter/a/67e7ee/32.png) [@anamika](https://community.openpbs.org/u/anamika)\ **Post date:** [October 19, 2017, 6:29pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/33 "2017-10-19T18:29:13Z")
@bhroam Edd looks good. I have one comment on the ‘estimated.soft\_walltime’ being writable by manager. I am not sure if current estimated.start\_time and estimated.exec\_vnode are settable by manager too. And if ‘estimated.soft\_walltime’ is settable then how would a manager can set it; via hooks or alterjob, etc … please clarify in the EDD. Also on other hand a bigger question is if we want manager/admins to set estimated.soft\_walltime values? ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [October 19, 2017, 8:19pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/34 "2017-10-19T20:19:58Z")
Good point @anamika. I looked into the code and estimated.start\_time and estimated.exec\_vnode can be modified by a manager. You can’t use qalter, but the API/hooks should be able to do it. The estimated attribute really is there so the scheduler can display data. Sites really shouldn’t be modifying it themselves. If they do, the scheduler is likely to overwrite it on the next scheduling cycle. It’s like the job’s comment. The site can change it, but it won’t stick. Should these three attributes explicitly be made read only except for the scheduler? I think it’s possible to do this. ---
**Author:** ![anamika](https://avatars.discourse-cdn.com/v4/letter/a/67e7ee/32.png) [@anamika](https://community.openpbs.org/u/anamika)\ **Post date:** [October 19, 2017, 9:26pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/35 "2017-10-19T21:26:49Z")
Thanks Bhroam for looking into it. I am not sure why those attributes were be allowed to be settable by manager at first place. @jon and @scc, can you comment if this was a business requirement? ---
**Author:** ![jon](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/jon/32/23_2.png) [@jon](https://community.openpbs.org/u/jon)\ **Post date:** [October 19, 2017, 10:16pm UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/36 "2017-10-19T22:16:57Z")
> [@bhroam](#): > > Should these three attributes explicitly be made read only except for the scheduler? I think it’s possible to do this. If it is possible without too much work than I think that would be that right thing to do. However, it is such a small corner case that I am not too worried about. ---
**Author:** ![anamika](https://avatars.discourse-cdn.com/v4/letter/a/67e7ee/32.png) [@anamika](https://community.openpbs.org/u/anamika)\ **Post date:** [October 20, 2017, 1:23am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/37 "2017-10-20T01:23:44Z")
Thanks Jon. May be we will not bother about fixing the existing attributes (estimated.start\_time and estimated.exec\_vnode) but we should be doing the estimated.soft\_walltime the right way meaning not allowing manager to set it. ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [October 20, 2017, 1:30am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/38 "2017-10-20T01:30:19Z")
I’ll fix all 3. It isn’t hard. I just have to set a few flags. The docs already say estimated.start\_time and estimated.exec\_vnode are read only. It’d only make sense that all 3 follow the same behavior. I’ll update the EDD as well. ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [October 20, 2017, 2:04am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/39 "2017-10-20T02:04:05Z")
I take it back, all 3 have to be manager settable. I can’t make estimated.start\_time and estimated.exec\_vnode writable only by the scheduler because of pbs\_est. It runs as a manager. I can’t make estimated.soft\_walltime writable only by the scheduler because soft\_walltime is the same resource as Resource\_List.soft\_walltime. If I make it only writable by the scheduler, a manager can’t set Resource\_List.soft\_walltime. I can’t change any of them. This doesn’t mean we should advertise that they are settable. If a manager does set them, the scheduler will overwrite them in the near future (like the next scheduling cycle). They’re kind of like the job’s comment. ---
**Author:** ![anamika](https://avatars.discourse-cdn.com/v4/letter/a/67e7ee/32.png) [@anamika](https://community.openpbs.org/u/anamika)\ **Post date:** [October 20, 2017, 6:27am UTC](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587/40 "2017-10-20T06:27:52Z")
Thanks for the explanation Bhroam and updating the EDD. [Previous page](https://community.openpbs.org/t/pp-482-non-destructive-walltime/587.md?page=1)