Doing logical NOT and/or OR in a -l select statement

We are bringing a new machine online with a dragonfly network. We have resources named tier0 which identify the racks (with a top of rack switch) and tier1 which identifies the dragonfly group the node is in. We are having trouble with one of the dragonfly groups. We don’t want to take the nodes offline because they are still functional for jobs that are not IO intensive, but we want to make sure things like IOR don’t end up on that dragonfly group. Ideally, what I would like to be able to do is something like:

-l select=32:tier1!=g2

so the nodes can be selected as long as they do not have tier1=g2. If I can’t do the above, and even if I can, it would also be nice sometimes to be able to do an or, so something like:

-l select=32:tier1=g0|g1|g3|g4

In that case the 32 nodes can come out of any of the listed groups.

I could not find anything like that in the manuals, so I suspect there is no support for it, but perhaps I missed something or there is a workaround?

Thanks,

Bill

Adding the or / the not equals operators in the select language could be a cool addition to PBS.

In absence of that, one possible way would be to add another resource to all the nodes - say something like IOR=true or false. Then you could set IOR=false for all the nodes under g2 and true for all the nodes under g0, g1, g3, g4. Then the select would look like:

-l select=32:IOR=true

Hi Bill,

Have you looked at the runone dependency? It allows you to submit a series of jobs with different requests, once one runs the rest are deleted. It allows you to do the OR that you’re asking about.

As for the NOT you could do as Subhasis suggests. Or, if you don’t want to touch all your nodes, you could create a simple run job hook that rejects IOR jobs that would run on the g2 nodes.

Sam Goosen

I don’t think the runone would get me what I want, which makes me think I didn’t explain it as well as I should have.

The specific example in this case is that we want to run a 256 node IOR, so what I was trying to do is tell the scheduler the pool of nodes I wanted it to select from.

So what I wanted to do is:

qsub -l select=256:tier1!=g2 ior.pbs

And I wanted that to tell the scheduler “you may choose any 256 nodes as long as tier1 does not equal g2”

or

qsub -l select=256:tier1=g0|g1|g3|g4|g5|g6|g7 ior.pbs

And that should mean “you can choose any 256 nodes as long as that node has tier1 equal to at least one of the listed values”

BTW, the way we got around it was this:

qsub -l select=32:tier1=g0+32:tier1=g1+32:tier1=g2+32:tier1=g3+32:tier1=g4+32:tier1=g5+32:tier1=g6+32:tier1=g7+32:tier1=g8 ior.pbs

I agree that expanding the select language could be useful. I added it to our list of possible features to add.