A standard package for PTL will be useful for all PTL users. As such, I am proposing two approaches for PTL packaging.
The first approach is .whl files as a pip enabled package.
The second one is .rpm packages which is currently being used by PBS to package itself.
Some notable points for each packaging mechanism.
Whl packaging
Rpm packaging
* Whl will handle dependencies through requirements.txt file
* Rpms will easily be able to handle all types of dependencies
* It is platform dependent
* Also platform dependent
* Existing rpm infrastructure present in PBS packaging
* Easier to implement than whl on our current infrastructure
* It will be easier to use by current users as they already use PBS rpms
As it suggests both methods are almost equivalent, except that we already have infrastructue for rpm packages. Contemplating on various pros on said packages I am propsing to go with rpm packaging over whl packaging. Building up on the existing directory structure we will create an additional rpm package for PTL, which will be compatible with the PBSpro package.
Please suggest any moderation/alternate approach if needed.
If we are focusing only on Rhel based Linux, its totally fine to go with RPM based solution.
But if we have any plan to provide common solution for Debain, Rhel/SLES and Windows platforms. I suggest to go with any one of Python’s de facto packaging solutions like 1) Wheel , 2) PyPI.
@hirenvadalia ya that’s true…
My forward thinking got that suggestion!
As I said above, Only if we have any plan to support windows. we should do for a common solution.
@bremanandjk I totally understand your forward thinking and I am also with you that if possible then definitely we should use common solution for all supported platforms.
And off course we have plan to add support for Windows in PTL.
But here we are taking about *nix vs Windows which is totally separate world that’s the reason we have separate packaging even in PBS itself (RPM/DEB for *nix and MSI for Windows)
Also here are few extra points which says it’s better not to use whl:
Whl adds extra dependency of installing pip in system which user/admin has to take care of it, while for RPM/DEB you don’t need any extra packages as its native package manager
If you want to keep whl platform independent then you have to generate something calling “Universal WHL” (aka noarch.whl) and universal whl is only possible and good (as far as I know) if you don’t have any C extensions in PTL. Which we don’t have as of now but we never know, in future we might add (for example like C extensions for PBS IFL API for PTL API Mode)
About Windows, we still don’t know what we will need on Windows to port PTL on it, there are lots of gray area which we need to fill it up first before we will able run PTL on windows (for example ~2 years back I tried to put PTL on Windows which has PowerShell extensions, C extensions, you can look at that code https://github.com/hirenvadalia/pbspro/tree/windows-ptl) so my point here is why to worry about part which we don’t know yet!
And if we use RPM/DEB right now then in future (Once we have Windows PTL) we can re-evaluate things and move to whl if it wins at that time, it’s a software development world where everything moves based on current and known upcoming future situations.
I suggest you name the package pbspro-ptl-VERSION.x86_64.rpm
If the package does not contain any compiled binaries, then you should change x86_64.rpm to noarch. Automake might actually do that for you.
Something to keep in mind… when PBS Pro is built for distribution with SUSE/RHEL we will not be allowed to place anything under /opt. We currently package as an ISV so /opt is the right place to put the files. We’ll have to support both scenarios. Perhaps we could add a new parameter to configure that would enable building the pbspro-ptl package. I suggest it be disabled by default. It could be enabled with something like… ./configure --enable-ptl
Looks good. Perhaps you want to mention some dependencies the package will have. Will PBS Pro itself be a prerequisite? These will appear as “Requires” lines in the spec file. Other than that, the design looks fine.
