Hi all,
One of the issues that has plagued me when trying to build tagged releases is that the build always winds up with a version number of xx.0.0, which never matches the tagged version number I’ve checked out. For example, if I check out and build v19.1.2, all commands return 19.0.0, pbs_version in $PBS_HOME says 19.0.0, and any RPMs I build are versioned at 19.0.0.
I propose we automate the process with a git hook that will update all of the appropriate files (configure.ac, pbspro.spec, etc.) when a new tag is added. The hook could also be expanded to do other things as well if you are manually doing things from tag release to tag release, such as building and updating binaries for download on pbspro.org.
I realize after digging through the files that you can override the version number with a configure option (./configure PBS_VERSION=19.1.2) but that is only documented in one place that I’ve found - configure.ac. Neither confluence nor pbspro/INSTALL document it. In any case, I would expect that overriding what is in configure.ac should be the exception in order to test or do a custom build, and not the rule.
Great suggestion. I think we only need to update configure.ac since pbspro.spec is generated based on pbspro.spec.in when configure is run. We also have to be careful not to break compatibility with OpenHPC. We have a different spec file in their repo that we maintain that hardcodes the version, so I don’t think it will cause problems.
Thanks for the proposal, I find the version numbers quite annoying as well, so it would be great to have a git hook to update the version numbers.
Having said that, I’m not sure how git hooks work, so I wanted to clarify, will the edits to configure.ac create a new commit that will get checked into the branch? Will the Github account of the user creating the tag be used to check that commit in? Do you know how this will work?
I have a different opinion about
we only need to update configure.ac since pbspro.spec is generated based on pbspro.spec.in when configure is run.
Because rpmbuild with -ta option sees SPEC file in SPEC directory even we do not extract explicitly,
appropriate pbspro.spec before configure is run is helpful.
18.1.4, 18.1.3 and so on are with appropriate configure.ac and pbspro.spec.
For unknown reason,
pbspro.spec files do not include ./autogen.sh
With ./autogen.sh in %build Section of pbspro.spec,
we can build RPMs like rpmbuild -ta SOURCES/pbspro-19.1.1.tar.gz
where
pbspro-19.1.1.tar.gz is source code
whose URL is https://github.com/PBSPro/pbspro/archive/v19.1.1.tar.gz
rather than partially processed file set named as pbspro-19.1.1.tar.gz
I think perhaps some of the output you provided was interpreted as markup, making your example difficult to follow. If you add three back ticks (```) before and after your example, it will become pre-formatted text.
Preformatted text
I don’t claim to be an expert with GNU autotools, and would be happy to consider your suggested changes. I just want to make sure I clearly understand what you are proposing.
We tried 19.1.2 and things are much better than 19.1.1.
19.1.2 seems fixed version nub
At least, version numbering is without any problems if we follow INSTALL(./autogen.sh, ./configure, make, make install).
As for building rpms,
./autogen.sh is still absent.
This may be caused
by policy which we do not know
or
by too late our submission
or
by our submission was interpreted as markup
We will write the same thing as last month.
With ./autogen.sh in %build Section of pbspro.spec,
we can build RPMs like rpmbuild -ta SOURCES/pbspro-19.1.2.tar.gz
where
pbspro-19.1.2.tar.gz is source code
whose URL is https://github.com/PBSPro/pbspro/archive/v19.1.2.tar.gz
rather than partially processed file set named as pbspro-19.1.2.tar.gz
guest@asusws00:~/Downloads$ md5sum pbspro-19.1.2.tar.gz v19.1.2.tar.gz
a25a3e21c20d977557a85fb58e70e65d pbspro-19.1.2.tar.gz
444c35b1647cd79ecfe19c459a948bea v19.1.2.tar.gz
[test01@cent7-16 SPECS]$ diff -c pbspro.spec pbspro.spec.orig
*** pbspro.spec 2019-07-16 14:17:41.522450866 +0900
--- pbspro.spec.orig 2019-06-12 23:11:21.000000000 +0900
***************
*** 268,274 ****
%setup
%build
- ./autogen.sh
[ -d build ] && rm -rf build
mkdir build
cd build
--- 268,273 ----
rpmbuild -ba SPECS/pbspro.spec.orig
[test01@cent7-16 rpm]$ rpmbuild -ba SPECS/pbspro.spec.orig
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.bvEyX6
+ umask 022
+ cd /homelocal/test01/rpm/BUILD
+ cd /homelocal/test01/rpm/BUILD
+ rm -rf pbspro-19.1.2
+ /usr/bin/gzip -dc /homelocal/test01/rpm/SOURCES/pbspro-19.1.2.tar.gz
+ /usr/bin/tar -xvvf -
((omitted))
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd pbspro-19.1.2
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.SbaPZb
+ umask 022
+ cd /homelocal/test01/rpm/BUILD
+ cd pbspro-19.1.2
+ '[' -d build ']'
+ mkdir build
+ cd build
+ ../configure PBS_VERSION=19.1.2 --prefix=/opt/pbs --with-pbs-server-home=/var/spool/pbs --with-database-user=postgres
/var/tmp/rpm-tmp.SbaPZb: line 33: ../configure: No such file or directory
error: Bad exit status from /var/tmp/rpm-tmp.SbaPZb (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.SbaPZb (%build)
I think I understand your point here, but let me restate to ensure that is the case. By adding a call to autogen.sh to the %build section of the spec file you may call rpmbuild without having to call autogen.sh before rpmbuild. Please confirm that is the motivation for the proposed change.
Assuming that is the motivation, I looked at a few of the spec files from the OpenHPC project. In a few cases I could see the build section does contain a call to autogen.sh, so your suggestion is not without precedent.
I would suggest you make the call conditional:
[ -f configure ] || ./autogen.sh
We will need to update the TravisCI build scripts along with this change. We also have an internal build infrastructure that we will need to validate prior to merge. Please open an issue in GitHub if there isn’t one already open. I see no reason why we should avoid your proposal. Does anyone else on the forum object?
Yes, the motivation is we may call rpmbuild without having to call autogen.sh before rpmbuild.
With autogen.sh, rpmbuild will succeeded against source code rather than partially processed pbspro-19.1.2.tar.gz
We were confused with the partially processed pbspro-19.1.2.tar.gz, https://github.com/PBSPro/pbspro/releases/download/v19.1.2/pbspro-19.1.2.tar.gz
, which seemed as if it were with special modification in source code and built RPMs(pbspro_19.1.2.centos_7.zip
) were with something special.
We checked
[ -f configure ] || ./autogen.sh
and confirmed it works as we expect.
We did not check the case where configure exist and autogen.sh is not executed.
We are not sure how we can keep “configure” file during rpmbuild.
Both rpmbuild -bc and rpmbuild -bp extract tarfile.
We tried followings but failed with the same error messages.
rpmbuild -bp SPECS/pbspro.spec.orig
cd BUILD/pbspro-19.1.2
./autogen
rpmbuild -bc SPECS/pbspro.spec.orig
I’m confused why you would want to keep the generated configure file during the rpmbuild process. Could you please explain why that is the case?
When I build a PBS Pro RPM I manually run autogen.sh, then configure, then “make dist” to generate the tar file used for rpmbuild. I then place the pbspro.spec in the SPECS directory and the source tarball in the SOURCES directory. The rpmbuild command then extracts the tarball and re-runs configure followed by “make install” to the staging directory used to create the actual RPM.
Perhaps you could describe the process you expect to follow when building PBS Pro so we can compare it to what is done today. Thanks for being patient, just trying to understand what you’re trying to accomplish.
For opening the GitHub issue, you are correct. Click on “New Issue” at the URL you provided.
Why we want to keep the generated configure file during the rpmbuild process is
to test
[ -f configure ] || ./autogen.sh
We could not understand why [ -f configure ] is required.
But with your helps, we understand that [ -f configure ] is required to avoid wasty autogen.sh.
a tarball generated by make dist keeps “configure” file.
We think we need to add [ -f configure ] || ./autogen.sh
to both pbspro.spec and pbspro.spec.in
Thank you for following up. Everything appears to be in order from my perspective. Please go ahead and file a pull request. I will run your dev branch through our build system to confirm nothing is broken prior to approving the PR.
You are correct about updating both pbspro.spec and pbspro.spec.in.
As for a pull request
we are not familar to github.
Anyway, we have submitted a pull request.
We are far from confident about what we did.
click “Fork”
git clone https://github.com/sstcdev/pbspro
git checkout -b issues/issue1205
vi pbspro.spec
vi pbspro.spec.in
git add pbspro.spec
git add pbspro.spec.in
git commit
git push origin issues/issue1205
Click “Compare & pul request”