bigtop
bigtop copied to clipboard
apache
BIGTOP-1561. Add Presto as a component
Presto has an RPM, it would be best to use that: https://github.com/facebook/presto/tree/master/presto-server-rpm.
Does it also have deb? Does the layout and the build process for this is done according to the Bigtop expectation (also used by Hadoop distributions at large)?
Oh, no; Presto doesn't have deb, only RPM. Is there anywhere that the Bigtop RPM expectations are documented?
There's no explicit document on RPM's requirements, but all of our packaging code is following the same rules and informal guidelines, codified within the existing packaging code. Good examples for complex packages would be hadoop and hbase. Lesser complex stuff is ignite-hadoop and similar.
We won't be able to accept only RPM packaging though: half of our supported OS are deb-based.
@cawallin I did basic packaging for presto to use our internal deployment and at that time, there is no packages like rpm or deb on Presto. I'm aware of that you and your team (Teradata, right?) are doing great work in Presto community :-) As of now, we have some prerequisites to integrate Presto into Bigtop stack. and... now, I believe it's a good starting point to discuss on that. Thanks!
Sure, thanks for the update, sorry I was a bit insistent before :) I'm not sure that we'll have room in our roadmap to work on deb packaging, but tools like presto-admin (https://github.com/prestodb/presto-admin) depend a little on how the RPM/init scripts work, and it'd be great if presto-admin worked for both RPMs. After looking more carefully through the code, I think it should work, though I made one comment about configurability (e.g. if you discover after installation that your /var mount is really small and want to change the log directory after the fact), and I'd have to test out the RPM with our presto-admin tests.
@cawallin I'll look into presto-server-rpm and presto-admin. I believe presto-admin is nice so it would be nice to have the compatibility between Bigtop's artifacts and presto-admin.
First of all, packaging rpm on Bigtop is a bit different from presto-server-rpm. overall, we have a base package and sub packages. the spec file for Hadoop is a good example: https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/hadoop/SPECS/hadoop.spec
I applied the rules to Presto's one. so there are base package 'presto' and the rest of the packages are sub package: 'presto-server' for server service, 'presto-cli' for Presto client, 'presto-jdbc' for JDBC driver and 'presto-doc' for generated documentation. usually, we have configurations and init script in the server package.
Looks like presto-server-rpm contains the server package only. I wonder if you have a plan for cli, doc and etc.
Thanks for the explanation! We aren't actively working on the RPM right now, but we have it in our backlog to look into adding the CLI/docs/verifier/benchmark driver to the RPM, though we'd probably have to refactor it to move to the packages/subpackages approach you have here.
On line 211 of install_presto.sh we reference the location of presto-env.sh without including the $CONF_FOLDER directory
