nessie icon indicating copy to clipboard operation
nessie copied to clipboard

Nessie GC module should depend on client.nessie.version after next Iceberg release.

Open ajantha-bhat opened this issue 2 years ago • 2 comments

As nessie gc module [in #3421] depends on Iceberg spark run time jar, it should depend on client.nessie.version similar to other modules.

It cannot depend on it now itself as client.nessie.version is 0.18.0 which doesn't contain the gc related client changes (like detached reference, etc). So, currently depending on the latest Nessie client. Once, Iceberg makes a new release, gc module has to be updated to client.nessie.version

note that if gc-module doesn't depends on client.nessie.version, if latest client breaks compatibility, gc module compile will fail.

ajantha-bhat avatar Mar 22 '22 14:03 ajantha-bhat

Also fallback to uber jar (iceberg-spark3-runtime) dependency while depending on the client.nessie.version

ajantha-bhat avatar Mar 23 '22 18:03 ajantha-bhat

Tried working on https://github.com/ajantha-bhat/nessie/commit/1bf78141b2e2f895d35bdcb417c3edc537916065

a) nessie-jaxrs-testextension and nessie-jaxrs-tests also probably has to depend on client.nessie.version along with client. b) Cannot depend on a single iceberg uber jar (iceberg-spark-runtime jar) because finally, the Nessie module in iceberg has to depend on nessie-gc module. So that single runtime jar from iceberg can support nessie gc too. (if not we get that shaded jackson annotation errors)

ajantha-bhat avatar Jul 27 '22 14:07 ajantha-bhat