Making 'job_sort_formula' a sched attribute

Developers
1

Hi,

I’m proposing making job_sort_formula a sched attribute instead of a server attribute to improve the utility of multi-sched configurations.

More details in the design doc:
https://pbspro.atlassian.net/wiki/spaces/PD/pages/1393721405/Making+job+sort+formula+a+sched+attribute

Please provide feedback, thank you!

  • Ravi
2

Might it be possible to set a server job_sort_formula as a default but then have an individual scheduler’s job_sort_formula override the server’s value?

3

Hey @smgoosen I think that if we want to deprecate server’s job_sort_formula then it’s better to make them incompatible. But, If we don’t want to deprecate server’s job_sort_formula and want to treat it similarly to server vs queue limits then yes, it would be valuable to do what you are suggesting. Please let me know what you think.

4

@smgoosen I don’t look at this like a server/queue limit. The job_sort_formula is a scheduling attribute. It existed at the server because we didn’t have a sched object at the time it was introduced. If we are going to move it to the sched object (where I think it really belongs), we should deprecate it at the server. With your suggestion, the only thing it really provides is a way to set some scheduler’s job_sort_formula individually and have the rest set to one single thing. We already have an easy way to set all schedulers at the same time (@default), and we don’t really expect there to be a huge number of schedulers at any one site do we? Is it really a hardship to set the job_sort_formula at every scheduler?

Bhroam

5

@agrawalravi90 your proposal looks fine to me. I do have some small comments -

  • I would suggest that you add the specific error message qmgr is going to throw when the admin is trying to set the formula on both the objects.
  • There is already an attribute called job_sort_formula_threshold which is operator settable, is there a reason this attribute isn’t?
6

Thanks for the feedback @arungrover. I’ve added the error message to the EDD.

I actually just copy pasted the permissions of job_sort_formula from the Ref guide. In the code, we actually say that job_sort_formula is READ_WRITE, so it’s actually writable by anybody today. Since ref guide is written according to intended design, I thought I’ll just follow that (and fix the code). What do you think?

7

@Bhroam I would agree with you so long as @default truly means all the schedulers - it doesn’t now, it only updates the default scheduler (see pbsdev discussion title “qmgr set @default behavior not consistent for different objects?” back in October).

@agrawalravi90 Re: permissions I would think job sort formula is a manager operation, otherwise it opens a huge hole for users to game the system for their own benefit. @Bhroam do you remember why it wouldn’t be restricted to admins?

8

I think that it is considered a bug, but we had decided that it was not really worth fixing. Maybe with this design change it’ll become more important to fix it? What do you think @scc ?

Our documentation does say that, but the code seems to provide read & write access to everyone:

9

Actually, I take it back, despite the attribute flag being incorrect, it gets protected because of how we transfer the job_sort_formula information:
[ravi@pbspro ~]$ qmgr -c ‘s s job_sort_formula=1*walltime’
qmgr: Failed to open /var/spool/pbs/server_priv/sched_formula for writing.
qmgr obj= svr=default: Unauthorized Request job_sort_formula
qmgr: Error (15007) returned from server

10

I was thinking we should be consistent in checking for permissions for both the attributes (formula and formula threshold). So either both the attributes have manager only write permissions or both the attributes have manager and operator write permissions.
I guess if we find out why is there this discrepancy it will help us decide which direction to take.

11

The meaning of operator and manager have not been kept over the years. Originally, managers set policy, where operators could handle the day to day operations. Therefore, managers create queues, but an operator could start or stop a queue or delete a job.

With that in mind, job_sort_formula and job_sort_formula_threshold should probably be manager only.

Bhroam

12

so, according to our documentation, both job_sort_formula_threshold and job_sort_formula are only admin settable. So, I can just fix the flags for both of them to be consistent with the documentation.