SRPM Output

[trevor-vaughan]

• Modify RPM build process to generate and collect SRPM and RPM artifacts by default
• Create a spec file that will allow for a natural rpmbuild --rebuild <thing>.src.rpm approach
• Ensure that required components are present in the RPM spec file
• At this time, not feature flagged. If users don't want the src.rpm output, they simply don't have to use it.
• Commented out Travis tests for EOL Rubies

Closes #237

[mathieu-aubin]

try building a npm package without setting any prefix and see if it fails due to file not found i had to add --prefix=/ to make it work if i remember correctly

[trevor-vaughan]

@mathieu-aubin I didn't change any settings except for those directly related to the RPM building.

[trevor-vaughan]

Also, I left in the support for Prefix but made it only exist if explicitly specified: https://github.com/jordansissel/fpm/pull/1657/files#diff-f32def54a1d6be460052b3c74f7aca7cR54

[mathieu-aubin]

yes, i see that. have you tried building a rpm of a npm package with your fork? When i do, it fails unless i set a prefix. On both fedora30 and 28.

will try in.a virtualenv

[trevor-vaughan]

@mathieu-aubin Can you post exactly how you're building the RPM? It's possible that we're just using two different methods and I was simply missing something during testing. I'll go ahead and update to add the Prefix in by default, simple enough and causes no issues except some warnings from rpmlint.

[mathieu-aubin]

# clone repo branch
git clone https://github.com/trevor-vaughan/fpm --branch=srpm-support && pushd fpm;

# install fpm frpm branch
make fpm-1.11.0.gem && gem install fpm-1.11.0.gem

# Failing build, without prefix
fpm \
	--output-type rpm \
	--input-type npm \
	--name webpagetest \
	--version 0.3.9 \
	--architecture noarch \
	--package webpagetest.v0.3.9-1.fc28.noarch.rpm \
	--rpm-summary='Webpagetest does web page tests' \
	--verbose --debug --debug-workspace \
	webpagetest

# Correct build, with prefix set to /
fpm \
	--prefix=/ \
	--output-type rpm \
	--input-type npm \
	--name webpagetest \
	--version 0.3.9 \
	--architecture noarch \
	--package webpagetest.v0.3.9-1.fc28.noarch.rpm \
	--rpm-summary='Webpagetest does web page tests' \
	--verbose --debug --debug-workspace \
	webpagetest

popd

i am thinking its got to do with npm prefix now... but just in case, can you double check? thanks

[mathieu-aubin]

npm config ls -l | grep prefix

gives me that /usr as prefix...

[mathieu-aubin]

I don't think that this is correct

%install
mkdir -p %{buildroot}%{prefix}
cp -r * %{buildroot}%{prefix}

that prefix (i believe) is used at install time in order to install the package in a certain directory

in fact, i just tested using a weird prefix and editing the spec file to correct thoses lines. Rpm installed into my weird prefix folder

[trevor-vaughan]

@mathieu-aubin Thanks for pinpointing this, should be corrected.

[trevor-vaughan]

@mathieu-aubin If possible, I'd like to get this through as functional and then have separate work for "pretty".

This PR doesn't address all of the ideas in the full comment thread, which are very good ideas, but I needed something that worked now and I'm OK with pretty later. Many thanks for your reviews, they've been really helpful!

[abitrolly]

@trevor-vaughan would be very nice to get Travis fix moved into separate PR to fix the repo first.

[mathieu-aubin]

This should be reviewed and merged

[sreerajkksd]

Can a maintainer take a look and merge this ?

[abitrolly]

@sreerajkksd unless somebody steps in to co-maintain, I doubt this. At the very least this PR should be rebased with no distracting changes like whitespace and commented tests.

[trevor-vaughan]

@abitrolly Rebased. Not interested in re-breaking trivial whitespace updates. Will happily add tests for detecting that a SRPM was output if you can point me at the tests that show how the systems detects that a RPM has been output.

[trevor-vaughan]

Test failures appear unrelated to this change.

[abitrolly]

Tests are not helping, because they are failing. https://github.com/jordansissel/fpm/pull/1681 contains fixes, and there are again no resources to review and mere that.

[jordansissel]

The tests aren’t in great shape these days, and there’s a few tests that always fail. For this PR, don’t worry about them.

On Sun, Mar 28, 2021 at 2:31 AM Anatoli Babenia @.***> wrote:

Tests are not helping, because they are failing. #1681 https://github.com/jordansissel/fpm/pull/1681 contains fixes, and there are again no resources to review and mere that.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jordansissel/fpm/pull/1657#issuecomment-808871291, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABAF2UC7VW6OM4MM46F35DTF3ZQNANCNFSM4IHIIM6Q .

[ryysud]

Can a maintainer merge this ? cc: @jordansissel

[ryysud]

@jordansissel If you prefer to re-create PR that conflicts are resolved, I can do it. What do you think?

[trevor-vaughan]

I can rebase this again. I didn't realize there was still interest in it to be honest.

[trevor-vaughan]

@ryysud and @jordansissel Rebased and removed vestiges of travis material.

[jordansissel]

Thank you for working on this — I still want to play with it and get you feedback but haven’t made time yet.

On Wed, May 18, 2022, at 5:24 AM, Trevor Vaughan wrote:

@ryysud https://github.com/ryysud and @jordansissel https://github.com/jordansissel Rebased and removed vestiges of travis material.

— Reply to this email directly, view it on GitHub https://github.com/jordansissel/fpm/pull/1657#issuecomment-1129940769, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABAF2WYRUFXEHO3TXIUY5LVKTOQPANCNFSM4IHIIM6Q. You are receiving this because you were mentioned.Message ID: @.***>

[trevor-vaughan]

@jordansissel No problem. Just update the thread if you find any issues.

[jordansissel]

Tested, but still want to review the code to see what you needed to do to make this happen <3

# Turn 'rails' rubygem into an rpm
% bundle exec bin/fpm -s gem -t rpm rails
Created package {:path=>"rubygem-rails-7.0.3-1.noarch.rpm"}

# Check for the srpm
% ls rubygem*.src.rpm
rubygem-rails-7.0.3-1.src.rpm

# Build from the srpm
% rpmbuild --rebuild --target noarch rubygem-rails-7.0.3-1.src.rpm

% ls ~/rpmbuild/RPMS/noarch/rubygem-rails-7.0.3-1.noarch.rpm
/home/jls/rpmbuild/RPMS/noarch/rubygem-rails-7.0.3-1.noarch.rpm

# Check files in the result of rpmbuild
% rpm -qlp ~/rpmbuild/RPMS/noarch/rubygem-rails-7.0.3-1.noarch.rpm
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/build_info
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/cache/rails-7.0.3.gem
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/doc
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/extensions
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/gems/rails-7.0.3/README.md
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/plugins
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/specifications/rails-7.0.3.gemspec

# Compare to the rpm that fpm created
% rpm -qlp rubygem-rails-7.0.3-1.noarch.rpm
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/build_info
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/cache/rails-7.0.3.gem
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/doc
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/extensions
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/gems/rails-7.0.3/README.md
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/plugins
/home/jls/projects/fpm/vendor/bundle/ruby/3.0.0/specifications/rails-7.0.3.gemspec

Nice!

My initial feedback is that I wonder if this should be a separate output type? Like --output-type srpm ?

The reason I ask is that historically the rpm type has output exactly one file, and multiple files might be a kind of confusing change to folks who aren't expecting it.

Additional thoughts:

Folks have asked for SRPM support for years, and is this what they were asking for? It basically turns rpmbuild into a tarball -> rpm conversion tool without any meaningful "build" step. This might be ok, and is very likely what I would use it for if I had to use a system that required SRPMs. What do you think?

Also, the resulting .src.rpm file doesn't respect the -p flag (which sets the output filename). For example:

% bundle exec bin/fpm -s empty -t rpm -n example -p my-fancy-package.rpm
Created package {:path=>"my-fancy-package.rpm"}

% ls example-1.0-1.src.rpm my-fancy-package.rpm
example-1.0-1.src.rpm  my-fancy-package.rpm

Any thoughts on what to do for this case?

[jordansissel]

Another thought: For rpmbuild --rebuild, one must supply the --target noarch for noarch packages such as pure Ruby or Python code which doesn't have any CPU-specific architecture code. Is there a way we can set the default cpu/os target?

I wonder, could we include %_target, %_target_cpu, and %_target_os macros in the rpm spec?

[trevor-vaughan]

My initial feedback is that I wonder if this should be a separate output type? Like --output-type srpm ?

I hadn't thought about users that wouldn't expect two RPMs to be output.

Adding a flag for the SRPM makes sense to me but it seems like you'd want to go ahead and generate it along with the main RPM to keep the logic changes small. Maybe just a --srpm flag so that you know you're going to get one output?

Folks have asked for SRPM support for years, and is this what they were asking for? It basically turns rpmbuild into a tarball -> rpm conversion tool without any meaningful "build" step. This might be ok, and is very likely what I would use it for if I had to use a system that required SRPMs. What do you think?

Well, it's the best you can get when you're not actually building from source 😄. I think it's a reasonable 80% case and actually works pretty well if you're packaging up something like a bunch of interpreted code.

The main benefit is that it gives you something that you can manually hack on later in case you want to do something more fancy than you can easily support with fpm. The other benefit is for places that simply require a SRPM for whatever reason 🤷.

Also, the resulting .src.rpm file doesn't respect the -p flag (which sets the output filename). For example:

That should be relatively easy to fix with some minor hackery. I'll work on getting it updated (hopefully in the near future).

I'll happily accept PRs onto the original if anyone else wants to help get it in there.

I wonder, could we include %_target, %_target_cpu, and %_target_os macros in the rpm spec?

Yeah, that shouldn't be too difficult.

[jordansissel]

I think it's a reasonable 80% case

Agreed

The other benefit is for places that simply require a SRPM for whatever reason 🤷.

Yes! When I think about folks requesting this feature, just having fpm output an SRPM at all would be a nice step forward for environments where a deployment step requires SRPMs as input (for whatever reason).

If I can make time, I'll send you some PRs for the changes listed in your above comment 👍

[jordansissel]

I wonder, could we include %_target, %_target_cpu, and %_target_os macros in the rpm spec?

Works! I've got this PR forked into a local branch and am working on it.

[jordansissel]

Adding a flag for the SRPM makes sense to me but it seems like you'd want to go ahead and generate it along with the main RPM to keep the logic changes small

I'm mostly focused on the behavior of the fpm command line. We can make the code simple and still have two different target packages (--output-type srpm, etc).