lakeFS icon indicating copy to clipboard operation
lakeFS copied to clipboard

Discovery: do we want to publish the assembled Spark metadata client to Maven central

Open talSofer opened this issue 3 years ago • 1 comments

Currently, we only publish the non-assembled Spark metadata client to Maven central. This task is to do some discovery and decide whether we want to publish the assembled jar too.

talSofer avatar Jul 27 '22 08:07 talSofer

We also do the exact opposite on lakeFSFS: there we publish assembled and plain JARs. At the very least we need to be consistent 😮

arielshaqed avatar Jul 27 '22 08:07 arielshaqed

In fact we publish (only) assembled metadata clients for GC (users need only add S3 / Azure hadoop plugins, depending on what they intend to use). Unassembled versions will be too hard for most users to get working.

Spark client is used almost exclusively for GC, which is generally performed by admins (or lakeFS.Cloud, of course).

arielshaqed avatar May 08 '23 09:05 arielshaqed