How to chose the vnode just only with RHEL7 or SUSE 11?

[Ve0]

Hello!
I have the PBS cluster with multiple OS, like RHEL5,6,7, Suse 11, 12.
I would lie to know, how can users chose just only RHEL 5 or 6 but exclude other ones? or chose RHEL 6 or SUSE 11, but exclude other OSs?

[adarsh]

Please follow this link ( instead of colors you use the flavour of the operating system )

Is it possible to submit a job to one of the nodes from a list of nodes?

++ Mike + Anne Please follow the below instructions , this might satisfy your requirement Define the custom resource: qmgr –c "c r my_node_type type=string_array,flag=h" Append the custom resource to resources: line in sched_config file: resources: “ncpus, mem, arch, host, vnode, aoe, my_node_type” Note: after updating sched_config file, it is necessary to kill -HUP the pid of pbs scheduler. Set each queue’s default_chunk for the new resource to the value you are using to associate…

[Ve0]

Thnk you for your answer, but this one in not exactly what I need.

I have the resource: osv
this resource depend on vNode OS. I have some different version of os in cluster: R60 (rhel 6.0), R65 (rhel 6.5), C60 (centos 6.0), C70 (centos 7.0), S12 (suse 12), S11 (suse 11)

I assign it on node depens on OS instaled

s n node1 resources_available.osv=R60
s n node2 resources_available.osv=R65
s n node3 resources_available.osv=C70
s n node4 resources_available.osv=S12

I need the solution for submitting the job with mention couple resources and cluster find that resource in cluster and push the job there. Example how it looks in LSF:

bsub -R “osv==R60||osv==R65” sleep 60

It means cluster will find the vnodes with osv=R60 or osv=R65 and push it on first free vnode, but will skip all free nodes with C70 and S12.

how can I do that in PBS?

[adarsh]

Please follow this link:

PP-506,PP-507: Add support for requesting resources with logical 'or' and conditional operators

I just wanted to minimize the overhead of having more than one job from a job_set on the calendar. If scheduler adds more than one job then scheduler will reserve those resources and likely not run some other job which it could have. I get your point too, I will take this off the document I’ll make this change in the document. It would really make things easy if we delete jobs as soon as one starts. I’ll keep this part of the change as “Experimental” since it can change based on feedback. I…

[bhroam]

@Ve0

I have the resource: osv
this resource depend on vNode OS. I have some different version of os in cluster: R60 (rhel 6.0), R65 (rhel 6.5), C60 (centos 6.0), C70 (centos 7.0), S12 (suse 12), S11 (suse 11)

If you turn osv into a string_array resource, you can create combined types. You can set osv on the RHEL6 machine to resources_available.osv=R60,R60_R65
You can set it on the RHEL6.5 machine to resources_available.osv=R65,R60_R65.

When someone wants R60 or R65, they request it. If they want either or, they request R60_R65.

@adarsh is correct, we are in the process creating a more generic specification that you can make requests with boolean logic. Until we finish that, this trick with the string array should help

Bhroam

[Ve0]

Yes, I can. for example: for 4 os versions mthe string will be:
R6,R6_R7,R5_R6,R7_R6,R6_R7,R5_R6_R7,R5_R7_R6, and etc…

And the string have to much. So I have to put all variants…

That is crazy!!!

[adarsh]

Agree , it is bit of an inconvenience . The team and community are working to enable the feature in the future release.

[Ve0]

I know (from secret agent in Altair) this request was 2 years ago for implementing this future… so? Any updates from the TEAM and Community?!?

[mkaro]

The functionality you’re requesting is being tracked by the following tickets:
https://pbspro.atlassian.net/browse/PP-506
https://pbspro.atlassian.net/browse/PP-507

Design documents may be found here:
https://pbspro.atlassian.net/wiki/spaces/PD/overview

These are among a short list of tickets that are given attention on a regular basis. Syntax of resource requests and scheduler performance are two main concerns. Once our current release cycle is complete, we will return our attention to this project.

If there are aspects of the current design that do not meet your needs or could be improved, we welcome your input.

[Ve0]

Created: 11/Nov/16 6:50 AM
Created: 11/Nov/16 6:54 AM

Any updates after that date?

You just confirm my words about 2 years ago

[mkaro]

The latest commercial release of PBS Pro (version 18.2.1) happens today, completing our release cycle. We will be focusing more attention on these tickets in the near future. There have been some internal discussions on the syntax qsub will accept, but we have not yet reached a consensus. We would be happy to consider any suggestions you may have. We’re looking for something intuitive that doesn’t conflict with shell syntax and maintains backward compatibility.

[Ve0]

May I know why you start looking in to it after 2 years? What did you do before? Why you did not investigate it at least 1 year ago?

I really worried about you support. I will create the ticket, and you will look into it after 2 years… this is scary.

[billnitzberg]

Hi @Ve0,

Thank you for expressing your interest in this feature; I know it can be frustrating to see a desired feature languish, especially in a case like this, where work was started, but then it was set aside due to higher priorities.

This feature was initially started by the Altair team, and got to the prototype stage, but no final design was ever completed. The team tries really hard to correctly prioritize new features based on overall customer demand and impact. As you might imagine, there are always a lot of potential new features, and the reality is that this one just hasn’t bubbled up to the top again (yet). I hope the Altair team will be able to focus on it again soon, but, again, that depends on priorities.

In this case, since the design for the feature is not done, you can help move it forward. It would be helpful if you could describe your use case – not just how you could use the feature or what implementation changes you’d like to see in PBS Pro, but really explain your high-level goal(s). For example, I want to be able to allocate either RHEL or SLES nodes is still a bit low level; better would be if you could also explain why? How will this help your users or your site? Understanding the high level use cases really helps achieve a good design.

If you’re interested in contributing to the coding of the feature, even better – we’d love to have your help – just say the word!

Thanks again for your feedback,

- bill

[Ve0]

Any updates on this issue?

[billnitzberg]

Hi @Ve0 – I was wondering if you could help by explaining your use case more, e.g., from above:

@billnitzberg

In this case, since the design for the feature is not done, you can help move it forward. It would be helpful if you could describe your use case – not just how you could use the feature or what implementation changes you’d like to see in PBS Pro, but really explain your high-level goal(s). For example, I want to be able to allocate either RHEL or SLES nodes is still a bit low level; better would be if you could also explain why? How will this help your users or your site? Understanding the high level use cases really helps achieve a good design.

Thanks!

[Ve0]

For example, a user needs to submit the job on the host with RHEL60 or RHEL65 (he’s job will run on both OS with no issue, and to catch the host with 2 OS will be faster), but he DON’T WANT to run the job on hosts with SUSE and RHEL50. Cluster have multiple OS and the user’s job is sensitive to OS. How can the user do that?

[billnitzberg]

Thanks – I can see how a general “either/or” facility would allow for more freedom in choosing nodes, hence potentially faster turn-around and/or better utilization. There is still discussion happening on PP-506/507, but nobody is working on an implementation at this time. It would still be useful to know more about your target environment, workload, application mix, and performance. If you can share more, it would be helpful both to help refine the design and also to explore performance implications of possible implementations, when that time comes.

Regarding the use case you described specifically, it seems that Bhroam’s suggestion for using string arrays may be a good choice. On the nodes, you’d define osv as a string array with both the general and specific OS choices, e.g., on a node running Red Hat Enterprise Linux v6.5, you’d define:

osv = RHEL, RHEL6, RHEL65

other nodes would be defined similarly, e.g.,

osv = RHEL, RHEL6, RHEL60
osv = RHEL, RHEL5, RHEL51
osv = SLES, SLES11, SLES111

And then, in your example, the job would request “osv=RHEL6”, which would match either the first or second node above (but not the others).

There could be other (even better) ways of doing this today, but it would depend on knowing more about your workload, the mix of jobs, why jobs need RHEL60 or RHEL65, but not RHEL5, etc. (For example, if the deciding factor is some version of some library, it might be better to set up a string array that lists the libraries, not the OSes).

Thx again!

[Ve0]

OK, what you will tell me if user wants RHEL6 and RHEL7 but NOT RHEL5? Any solutions?

[billnitzberg]

Can you say more about why? Is it a library requirement? Is it a particular application requirement? What is causing the job to require RHEL6 or RHEL7, but not RHEL5? It may be that what is driving the choices will be easier to describe than the choices themselves… For example if there are 5 different math libraries, and they each work on only a specific set of OSes, then it would be better to have a setting for the library, e.g.,

mathlib = XYZ

and have jobs request nodes that support “mathlib=XYZ”.

Again, in this very specific case, it’s easy again. Just add something like RHEL6_PLUS, so the list becomes:

osv = RHEL, RHEL5, RHEL5_PLUS, RHEL51
osv = RHEL, RHEL6, RHEL5_PLUS, RHEL6_PLUS, RHEL60
osv = RHEL, RHEL6, RHEL5_PLUS, RHEL6_PLUS, RHEL65
osv = RHEL, RHEL7, RHEL5_PLUS, RHEL6_PLUS, RHEL7_PLUS, RHEL70
osv = SLES, SLES11, SLES11_PLUS, SLES111

And the example job asks for “osv=RHEL6_PLUS”.

For a specific workload at a specific site, there usually aren’t that many choices, so this approach can work well in most cases. There are always exceptions, however… How much value comes from handling those exceptional cases depends on how … exceptional … they are. Again, would be very helpful to have a full understanding of the target workload, system, etc. to understand better.

Thanks again.

[Ve0]

OK) We already have a pretty long string. Let’s make it more complicated))))

What about submitting the job for RHEL5 and RHEL6 but not RHEL7 and SLES11 for the first user and second user will submit just only RHEL6 and RHEL7… And another user will submit job only for RHEL6 and SLES11.

Please, provide me the string for this set up, if you can)))) HAHAHAHA)))

This is an example just only for 4 kind OS in a cluster. What about if I have RHEL5, 6, 7, SLES10, 11, 12, UBNT1404, 1804, AIX and more…

Man, just give up.