Adding support for cgroups

Developers
1

This is to inform the community work being done to support cgroups. Details may be found here: https://pbspro.atlassian.net/wiki/display/PD/PP-325+Support+Cgroups

Please feel free to provide feedback on this topic.

2 Likes
2

Hi jon,
I want to try out cgroups but I miss the cgroups.py file. Could you please provide an example or give me a hint?

3

We are in the process of reviewing the merge. The files should be available soon.

4

vchlum,

I have submitted a pull request to merge the cgroup work into open source. If you would like early access to it you can go look at the pull request and copy the pbs_cgroups.json and pbs_cgroups.py file.

For some help on getting started please refer to https://pbspro.atlassian.net/wiki/display/PD/PP-325+Support+Cgroups

If you give it a try, I would be interested in your feedback.

Jon

5

And we would love early comments on the feature and the code itself from everybody. Please feel free to review the code and let us know what you think about it.

Regards,
Subhasis

6

Thank you for the files. It seems to work for me ok (tested on Debian 8).

Checking the get_vmem_on_node I discovered a typo.
On line pbs_cgroup.py:1341:
percent_reserve = config['cgroup']['memsw']['reserve_memory']
should be:
percent_reserve = config['cgroup']['memsw']['percent_reserve']

Percent reserve is also not mentioned on the wiki page.

7

vchlum,

You are correct that I did not mention percent_reserve. When we first looked at this we decided to do a percent_reserve as a way to reserve some RAM for processes other than PBS (i.e. the OS). However, this did not give us a very reliable number especially on systems where multiple types of nodes with different amounts of memory. I decided to use reserve_memory so that I could always reserve a specific amount. I will look into this and clean it up as time allows.

Thanks for you feedback.

8

Both percent_reserve and reserve_memory are valid options, though I see that percent_reserve was removed from the default configuration file. At the moment, it appears these are mutually exclusive. I believe they should be additive. There will be some changes to the hook over the next couple weeks, so please keep your feedback coming!