# PP-339 and PP-647:release vnodes early from running jobs **URL:** \ **Category:** Developers\ **Created:** [January 24, 2017, 9:04pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419 "2017-01-24T21:04:55Z")\ **Posts on this page:** 20\ **Page:** 4
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [June 23, 2017, 1:16am UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/61 "2017-06-23T01:16:45Z")
I can only see bad things happening if someone released a node that was part of a cray reservation. PBS would think it was free and cray wouldn’t. PBS would try and use it again and cray would reject it. We’d end up with a lot of held jobs. What should we do about it? I don’t know. I don’t think there are any good answers. The only one I see is what Lisa says. Look for “cray\_” and document that if you want to use the node ramp down feature, don’t put “cray\_” in any node’s vntype that you want to use the feature. Bhroam ---
**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\ **Post date:** [June 23, 2017, 3:06pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/62 "2017-06-23T15:06:08Z")
First, just an implementation idea as an alternative to checkfing for “cray\_” in vntype (which is not now mandated): can we have the server issue the release nodes request to the job’s primary execution host and have any pbs\_mom that is running with $alps\_client set reject the request? Interface 2 is silent on whether or not it is supported for jobs involving Cray XC nodes, and I think it needs to be explicit since it is wholly separate from interface 1 from a user/admin perspective (though of course the back end code is in common). I believe it would not work on such jobs at present since my understanding is that the ALPS reservation persists until after file stageout completes (I may be wrong about that, though, please feel free to set me straight if I am!). In the future I think it may be worth investigating if we can continue to disallow incremental shrinking of a job while it is still running for jobs running on Cray XC systems (since we can’t modify the ALPS reservation) but allow the “release\_nodes\_on\_stageout” feature to be used by destroying the ALPS reservation before file stageout (and also freeing the compute nodes in PBS of course). In my mind this may be possible to support since we know the job is done with the compute nodes by the time file stageout happens, whereas with interface 1 it can be called at any time in the job’s lifecycle. This would be out of scope for the current work in my opinion, though. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 23, 2017, 4:11pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/64 "2017-06-23T16:11:03Z")
> [@bhroam](#): > > What should we do about it? I don’t know. I don’t think there are any good answers. The only one I see is what Lisa says. Look for “cray\_” and document that if you want to use the node ramp down feature, don’t put “cray\_” in any node’s vntype that you want to use the feature. Your suggestion seemed reasonable. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 23, 2017, 4:20pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/65 "2017-06-23T16:20:47Z")
> [@scc](#): > > First, just an implementation idea as an alternative to checkfing for “cray\_” in vntype (which is not now mandated): can we have the server issue the release nodes request to the job’s primary execution host and have any pbs\_mom that is running with $alps\_client set reject the request? This is possible although, will cray login nodes also have $alps\_client set on the mom side or will only be for cray compute nodes? (@lisa-altair) think it’s best not to ramp down vnodes with vntype ‘cray\_compute’, ‘cray\_login’, or ‘cray\_compile’, and perhaps any vnode in the future with the ‘cray\_\*’ vntype. ---
**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\ **Post date:** [June 23, 2017, 4:38pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/66 "2017-06-23T16:38:05Z")
There is no pbs\_mom running on hosts that get represented as “vntype=cray\_compute”, those vnodes are represented and accessed through the pbs\_moms running on the hosts that are represented by the vnodes with vntype=cray\_login, where alps\_client must be set. (And remember those vntype values CAN be changed arbitrarily, those are just the default values). It is certainly possible (though as far as I know uncommon) for a cray\_login type node to serve as the primary execution host for a job that does not actually use any cray\_compute type nodes, where a node release would technically be possible but disallowed by this approach. If others see this as an unacceptable drawback please speak up. An unstated benefit to this proposed approach is it keeps all of the Cray-specific code in the mom, which is (I believe) still the only place we have anything truly specific to this platform. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 23, 2017, 5:45pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/67 "2017-06-23T17:45:12Z")
I see, ok this approach makes sense then. Should I explictly state in the EDD that for Cray X\* series nodes, it will be those managed by pbs\_mom with $alps\_client set, or is that too implementation detail? ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 23, 2017, 5:49pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/68 "2017-06-23T17:49:18Z")
> [@scc](#): > > Interface 2 is silent on whether or not it is supported for jobs involving Cray XC nodes, and I think it needs to be explicit since it is wholly separate from interface 1 from a user/admin perspective (though of course the back end code is in common). I believe it would not work on such jobs at present since my understanding is that the ALPS reservation persists until after file stageout completes (I may be wrong about that, though, please feel free to set me straight if I am!). ‘release\_node\_on\_stageout’ will have the same restrictions as in pbs\_release\_nodes. Ok, I’ll make it explicit in the doc. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 23, 2017, 5:51pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/69 "2017-06-23T17:51:14Z")
> [@scc](#): > > In the future I think it may be worth investigating if we can continue to disallow incremental shrinking of a job while it is still running for jobs running on Cray XC systems (since we can’t modify the ALPS reservation) but allow the “release\_nodes\_on\_stageout” feature to be used by destroying the ALPS reservation before file stageout (and also freeing the compute nodes in PBS of course). In my mind this may be possible to support since we know the job is done with the compute nodes by the time file stageout happens, whereas with interface 1 it can be called at any time in the job’s lifecycle. This would be out of scope for the current work in my opinion, though. Interesting thought… ---
**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\ **Post date:** [June 23, 2017, 6:49pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/70 "2017-06-23T18:49:23Z")
> [@bayucan](#): > > I see, ok this approach makes sense then. Should I explictly state in the EDD that for Cray X\* series nodes, it will be those managed by pbs\_mom with $alps\_client set, or is that too implementation detail? I’d like to hear from @lisa-altair and/or @bhroam (and maybe @mkaro) on their views of this implementation idea before we go much further, but as for how to state this in the EDD I’d like to be as precise as possible with something like “this interface is not supported for jobs which have a primary execution host running a pbs\_mom with $alps\_client set, and will return this error …”. ---
**Author:** ![bhroam](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bhroam/32/20_2.png) [@bhroam](https://community.openpbs.org/u/bhroam)\ **Post date:** [June 23, 2017, 7:17pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/71 "2017-06-23T19:17:49Z")
> [@scc](#): > > In the future I think it may be worth investigating if we can continue to disallow incremental shrinking of a job while it is still running for jobs running on Cray XC systems (since we can’t modify the ALPS reservation) but allow the “release\_nodes\_on\_stageout” feature to be used by destroying the ALPS reservation before file stageout (and also freeing the compute nodes in PBS of course) Interesting point. This would also work on a pbs\_release\_nodes -a as well (just not a partial release). > [@scc](#): > > I’d like to hear from @lisa-altair and/or @bhroam (and maybe @mkaro) on their views of this implementation idea before we go much further, but as for how to state this in the EDD I’d like to be as precise as possible with something like “this interface is not supported for jobs which have a primary execution host running a pbs\_mom with $alps\_client set, and will return this error …”. Cray’s are a funny beast. If a job doesn’t request a Cray mom node as their first chunk, they will be assigned one (but I don’t think it shows up as MS). While it is common practice that the login nodes are also the mom nodes, do they have to be? Could a site do something like put their mom on the sdb node? Should we be assuming the moms are on login nodes? As for what the EDD says, I’d use language we use in the docs. I seem to remember the term ‘inventory mom’ used somewhere. I might be misremembering though. Bhroam ---
**Author:** ![mkaro](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/mkaro/32/85_2.png) [@mkaro](https://community.openpbs.org/u/mkaro)\ **Post date:** [June 23, 2017, 7:19pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/72 "2017-06-23T19:19:58Z")
It’s probably sufficient to state that Cray compute nodes are excluded without being more specific. I assume that MAMU nodes will be supported. ---
**Author:** ![lisa-altair](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/lisa-altair/32/643_2.png) [@lisa-altair](https://community.openpbs.org/u/lisa-altair)\ **Post date:** [June 23, 2017, 9:50pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/73 "2017-06-23T21:50:33Z")
> [@scc](#): > > It is certainly possible (though as far as I know uncommon) for a cray\_login type node to serve as the primary execution host for a job that does not actually use any cray\_compute type nodes, where a node release would technically be possible but disallowed by this approach. If others see this as an unacceptable drawback please speak up. I’m okay with this. > [@scc](#): > > I’d like to hear from @lisa-altair and/or @bhroam (and maybe @mkaro) on their views of this implementation idea before we go much further, but as for how to state this in the EDD I’d like to be as precise as possible with something like “this interface is not supported for jobs which have a primary execution host running a pbs\_mom with $alps\_client set, and will return this error …”. This phrasing seems good to me. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 26, 2017, 7:00pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/74 "2017-06-26T19:00:12Z")
@scc: pbs\_release\_nodes will not allow a node to be released that is managed by the mother superior, which is the primary execution host. There’s a specific error for this that is mentioned in the EDD: EDD excerpt: pbs\_release\_nodes will report an error if any of the nodes specified are managed by a mother superior mom. Example: % pbs\_release\_nodes -j 241 borg[0] pbs\_release\_nodes: Can’t free ‘borg[0]’ since it’s on an MS host I can just add something where nodes that are tied to Cray X\* series systems are those managed by mom with $alps\_client set, and it will return the appropriate, existing message about not allowing sister nodes tied to Cray X\* sseries systems to be released. existing EDD excerpt: % pbs\_release\_node -j 253 cray\_node "pbs\_release\_nodes: not currently supported on Cray X\* series nodes: \" ---
**Author:** ![altair4](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/altair4/32/34_2.png) [@altair4](https://community.openpbs.org/u/altair4)\ **Post date:** [June 26, 2017, 7:33pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/75 "2017-06-26T19:33:30Z")
---
**Author:** ![scc](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/scc/32/234_2.png) [@scc](https://community.openpbs.org/u/scc)\ **Post date:** [June 26, 2017, 8:24pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/76 "2017-06-26T20:24:26Z")
Thanks, Al. I am not sure if this is what you are implying with your reply or not, but the primary execution host name of the job will appear in the Mom = line of all of the X\* execution hosts, so if that is the mechanism the code is using to determine whether or not a vnode is managed by a mother superior mom then it may already be rejected, no additional $alps\_client checking required. Even if this is already the case the additional check in the code and more explicit detailing in the EDD may still be beneficial, though. Also , while I re-read this, it stuck out at me that “since it’s on an MS host” is probably not what we want in the error message, instead I think we should use “primary execution host”. I think it is OK (though not ideal) to use the two terms interchangeably in the text of the EDD, but for actual log messages in the software I would very much prefer that we stick to “primary” and “secondary” rather than “MS” and “sister”. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 26, 2017, 9:40pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/77 "2017-06-26T21:40:24Z")
> [@scc](#): > > Thanks, Al. I am not sure if this is what you are implying with your reply or not, but the primary execution host name of the job will appear in the Mom = line of all of the X\* execution hosts, so if that is the mechanism the code is using to determine whether or not a vnode is managed by a mother superior mom then it may already be rejected, no additional $alps\_client checking required. Even if this is already the case the additional check in the code and more explicit detailing in the EDD may still be beneficial, though. > > Also , while I re-read this, it stuck out at me that “since it’s on an MS host” is probably not what we want in the error message, instead I think we should use “primary execution host”. I think it is OK (though not ideal) to use the two terms interchangeably in the text of the EDD, but for actual log messages in the software I would very much prefer that we stick to “primary” and “secondary” rather than “MS” and “sister”. Scott, yes I’m implying MS host to be primary execution host, and the one appearing as “Mom” vnode attribute value. I see in the admin guide that we interchangeably use “mother superior” and “primary execution host” as well as “secondary execution host” and “sister”, although, “secondary” is not really defined in our reference guide, and instead, we mention “subordinate mom”. I’ll keep the use of “mother superior” and “sister” in the EDD, but I’ll go ahead and change to “primary execution host” instead of “MS” in an error message to pbs\_release\_nodes when releasing a mother superior vnode. Here’s the definitions in the PBS reference guide: "Mother Superior Mother Superior is the MoM on the head or first host of a multihost job. Mother Superior controls the job, communicates with the server, and controls and consolidates resource usage information. When a job is to run on more than one execution host, the job is sent to the MoM on the primary execution host, which then starts the job. Moved Primary Execution Host The execution host where a job’s top task runs, and where the MoM that manages the job runs. Sister Any MoM that is not on the head or first host of a multihost job. A sister is directed by the Mother Superior. Also called a subordinate MoM. Subordinate MoM Any MoM that is not on the head or first host of a multihost job. A subordinate MoM is directed by the Mother Superior. Also called a sister. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [June 27, 2017, 8:10pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/78 "2017-06-27T20:10:38Z")
> [@altair4](#): > > I don’t understand why the -j \ circumlocution should be > > required. Is it not anticipated that releasing the sister nodes will be > > a very common usage case? If so, then why shouldn’t we be able to effect > > this using simply pbs\_releasenodes -a? The job identifier is readily > > available in the running job’s environment. You’re the second one who suggested not making ‘pbs\_release\_nodes -j ’ a required option, rather if it’s not given, it’s likely called inside a job where pbs\_release\_nodes can just get the jobid from $PBS\_JOBID environment variable. Initially, I didn’t want to do this because pbs\_release\_nodes may not just be applying to running jobs but also with reservations via a new option later, say -r. But I’m getting convinced that we should allow what you suggested. Unless someone objects, I’ll go ahead and make the EDD change. > [@altair4](#): > > regarding the > > MS will hold onto the data, and will be added during final aggregation of resources\_used values when job exits > > note - what happens to those data if the MS is restarted? As mom holds onto the data for the job, it will also be saved on disk in the internal job files. So if MOM is restarted, it will just recover the data form the job file. > [@altair4](#): > > what does partial release of vnodes may result in leftover cpusets mean? > > I can understand that we might not want to attempt to shrink a CPU set on > > a host using pbs\_mom.cpuset, but given that we always allocate a single, > > indivisible CPU set to a job on an Altix host running the CPU set MoM, I > > can’t see why there’s a difficulty in clearing all the vnodes from a CPU > > set-enabled MoM’s host. Is it perhaps just a matter of documenting that > > when releasing a CPU set-enabled MoM from a job, the host name must be > > specified rather than the individual vnode name(s)? Yes, the trouble is in releasing individual vnodes managed by a cpuset mom. We can enhance pbs\_release\_nodes to work with cpuset-ed moms on the next release. It’s not targeted for this initial version. > [@altair4](#): > > is there information missing after the > > those vnodes that are part of a cgroup would not get automatically released > > note? Should an until the entire cgroup is released clause be added? Or …? Yes, I think the clause “until the entire cgroup is released for the job” should be added. > [@altair4](#): > > should the pbs\_release\_nodes: Unauthorized Request message include information > > about the originator of the attempted action? The information can be obtained from the server\_logs much like how it works with other other PBS commands like qrun executed by non-root. This reminds me, I need to put in the EDD that if pbs\_release\_nodes fails with “Unauthorized User”, then server\_logs would show the message like: 6/27/2017 15:13:45;0020;Server@corretja;Job;15.corretja;Unauthorized Request, request type: 90, Object: Job, Name: 15.corretja, request from: [pbsuser@corretja.pbspro.com](mailto:pbsuser@corretja.pbspro.com) > [@altair4](#): > > nit: use of these when referring to a single item (as in the example of freeing a single node - lendl249) > > these nodes are not part of the job: lendl) is bad English Good point. I’ll replace “these nodes” with “node(s)” so it can be applied to both single nodes or multiple nodes lin the list on the error message. > [@altair4](#): > > regarding the use of pbs\_release\_nodes on certain Cray or CPU set-enabled > > systems: is this check made first (good) or after having partially > > completed the task of releasing nodes (bad)? As with the other cases, all vnodes specified in pbs\_release\_nodes must be releasable, but if one fails, like a Cray check, then none gets released. > [@altair4](#): > > why must pbs\_release\_nodes always be verbose? Can it not be given a > > flag that says to report success or failure only via exit status? Yes, that would be a nice option. We can add this enhancement on the next release of node ramp down feature. > [@altair4](#): > > so At every successful pbs\_release\_nodes call, qstat will show […] > > implies that pbs\_release\_nodes always also causes qstat to be called? > > Ick. This seems like a choice that ought to be left to individual > > sites’ discretion. Of course qstat will not be called automatically by pbs\_release\_nodes! It’s not meant to be implied that way. > [@altair4](#): > > regarding > > Note that the execjob\_end hook will not execute on the host in this case: > > this seems unkind to any sites that have actions embedded in such a hook > > and now might have to choose whether to relocate whatever was in the hook > > or forego pbs\_release\_nodes. The release vnode early request from pbs\_release\_nodes (i.e. IM\_DELETE\_JOB2) is different from a normal delete job request (i.e.qdel/IM\_DELETE\_JOB) as the former happens and the entire job has not ended yet, whereas the latter, the job is at the end. So the former, execjob\_epilogue executes while the latter the execjob\_end hook. So yes, sites will be made aware of this via our documentation. > [@altair4](#): > > regarding the pbs\_relnodesjob() API: it’s safer for the extend parameter > > to be of type void \* rather than char \* - we are not prescient and cannot > > anticipate what we might want to eventually use it to convey. I’m just being consistent with the other PBS api functions. All the ‘extend’ parameters are of “char \*” type. Perhaps it will be an infrastructure project in PBS later to convert all the types to “void \*”. > [@altair4](#): > > nit: in the line following > > tail -f /var/spool/PBS/server\_priv/accounting/201701231, > > an 01/ has been omitted from the log message date stamp; also note that Ok, will fix this. > [@altair4](#): > > it would be helpful to describe (or highlight) the differences the user > > should see between the two tail -f invocations I’ve actually highlighted them in different colors blue, green, red… > [@altair4](#): > > does > > The ’c’ accounting record will show the next assigned exec\_vnode, > > exec\_host, Resource\_List. along with the job attributes in the > > new/next phase of the job > > imply that as MoMs are added to a job’s sisterhood and confirm their > membership with the Mother Superior, there’s a new record for each > of the next assigned exec\_vnode\<, exec\_host […]? I assume not, but > please clarify this. No, it’s only when there’s a release nodes action. I’ll need to clarify that the new accounting records appear as a result of release node action. > [@altair4](#): > > interface 5 uses the term phased job for which I see no definition; > is this intended to be a term of art? Yes, this needs to be defined exactly in the EDD. > [@altair4](#): > > interface 5 also refers to the log parser. Is this intended to refer to > a PBS facility or generically (for the latter, logging via syslog would > be one such example of many log parser back ends)? If the latter, that > may be quite a burden that’s being imposed on sites. Refers to a PBS facility. > [@altair4](#): > > interface 5 ends with showing the job’s values in total at the end of the job:. > Nothing follows that terminating colon. It supposed to be a period (.). I’ll fix. > [@altair4](#): > > regarding interface 6: will the message always be > Node\;deallocating 1 cpus from job \ - that is, > will it always report one CPU? If so, don’t pluralize; if not, use plural only when appropriate. I’ll change it to “cpu(s)” so it can be applied to both cases. > [@altair4](#): > > interface 7 would likely be more helpful if it noted what attributes were > available rather than the unhelpful > Some example internal job attributes […] are It’s a private, experimental interface, showing some internal attributes that may be added to later. I’ve actually listed what’s there so far, but it could get added to later. ---
**Author:** ![altair4](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/altair4/32/34_2.png) [@altair4](https://community.openpbs.org/u/altair4)\ **Post date:** [June 27, 2017, 9:37pm UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/79 "2017-06-27T21:37:48Z")
It might be good to raise the issue of the two terms having very similar meanings with our documentation department. Seems like some explanation about usage context might help. ---
**Author:** ![bayucan](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/bayucan/32/24_2.png) [@bayucan](https://community.openpbs.org/u/bayucan)\ **Post date:** [July 4, 2017, 1:15am UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/80 "2017-07-04T01:15:04Z")
> [@bayucan](#): > > @scc First, just an implementation idea as an alternative to checkfing for “cray\_” in vntype (which is not now mandated): can we have the server issue the release nodes request to the job’s primary execution host and have any pbs\_mom that is running with $alps\_client set reject the request? In thinking about this some more and trying it code-wise, this goes a different route as to how node ramp down is currently implemented. The pbs release nodes request is sent to the server, and the server figures out which sister nodes are allowed to be released and does so, modifying the appropriate internal attributes and structures, and then telling the primary execution mom that this is how assigned vnodes look now and to go ahead and update your own internal tables/structures. It’s going to be a major change to move this logic now entirely on the primary mom side, and also many issues, subtleties arise in doing that. So I’ll have to go back to Bhroam’s proposal to look for the “cray\_” string in the vnode’s vntype value to determine that vnode is not allowed to be released. This will be added to the EDD to replace the note about “vnodes managed by mom with $alsp\_client set in config file”. ---
**Author:** ![sgombosi](https://yyz2.discourse-cdn.com/flex030/user_avatar/community.openpbs.org/sgombosi/32/50_2.png) [@sgombosi](https://community.openpbs.org/u/sgombosi)\ **Post date:** [July 5, 2017, 2:51am UTC](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419/81 "2017-07-05T02:51:17Z")
Rather than have the releasability of a node special-cased for certain hard-coded vntypes, why shouldn’t releasability be a vnode attribute all by itself? [Previous page](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419.md?page=3) [Next page](https://community.openpbs.org/t/pp-339-and-pp-647-release-vnodes-early-from-running-jobs/419.md?page=5)