nessie icon indicating copy to clipboard operation
nessie copied to clipboard

Creating/dropping the namespace doesn't capture committer and author information.

Open ajantha-bhat opened this issue 1 year ago • 12 comments

Commit log entry: (doesn't contain author and committer info)

[LogEntry{commitMeta=CommitMeta{hash=06f3272a0448bcc177f80313d19b1315973e918637b3bff6c3262b588a25bedf, committer=, author=, allAuthors=[], signedOffBy=null, allSignedOffBy=[], message=delete namespace ns, commitTime=2023-04-05T06:29:58.785Z, authorTime=2023-04-05T06:29:58.785Z, allProperties={}, parentCommitHashes=[]}, additionalParents=[], parentCommitHash=b0911d20316931f42ce9503118f78164771fa00e7edbef4a5f4af5e65bf8ddb0, operations=null},

Should these commits happen from NessieIcebergClient or pass the committer and author info to the server? I am in favour of committing from NessieIcebergClient to have uniform behaviour with other commits.

https://github.com/projectnessie/nessie/blob/ad6d360f5fa57f99e998667d9436fddaf3b11cf0/api/client/src/main/java/org/projectnessie/client/util/v2api/ClientSideCreateNamespace.java#L89-L94

ajantha-bhat avatar Apr 05 '23 06:04 ajantha-bhat

cc: @dimas-b, @snazy

ajantha-bhat avatar Apr 05 '23 06:04 ajantha-bhat

Normally, committer is derived from AuthN data. Clients cannot set it.

dimas-b avatar Apr 05 '23 16:04 dimas-b

IIRC, the Iceberg-side Nessie code should already be setting author data... I may be mistaken, though. @ajantha-bhat : Could you double check?

ClientSideCreateNamespace is more of a backward compatibility fixture to simplify migration from API v1 to v2, IMHO.

Ideally, API v2 clients should use the commitMultipleOperations() method directly in order to create namespaces, and set proper author data.

dimas-b avatar Apr 05 '23 16:04 dimas-b

I have mentioned the code snippet above where we are not setting the commit meta with author and other information.

For a normal commit like create table, Iceberg side we add commit meta like this

https://github.com/apache/iceberg/blob/d485cc83ec58bd587fbfaaacebcf624c7f4c44d3/nessie/src/main/java/org/apache/iceberg/nessie/NessieUtil.java#L65-L91

ajantha-bhat avatar Apr 05 '23 16:04 ajantha-bhat

Yes, I edited the above comment a bit after posting it :)

dimas-b avatar Apr 05 '23 16:04 dimas-b

yes. My question is commitMultipleOperations() is called at client side for create table, commit table, etc.

So, should we also have commitMultipleOperations() to create a namespace at the client side? So, it will also handle the author and committer info.

ajantha-bhat avatar Apr 05 '23 16:04 ajantha-bhat

Re: the java client API, it might be worth adding an input method for CommitMeta to CreateNamespaceBuilder and similar builder. The new method can be supported only in API v2 (ignored in v1).

dimas-b avatar Apr 05 '23 16:04 dimas-b

So, should we also have commitMultipleOperations() to create a namespace at the client side? So, it will also handle the author and committer info.

This is preferable, I believe. REST API v2 does not have a dedicated endpoint for namespace manipulation. All changes should be done via the commit API, so the less wrappers we have around that, the better, IMHO.

dimas-b avatar Apr 05 '23 16:04 dimas-b

Ack, I will take a look at adding commitMultipleOperations() at the client side and cleaning up the code for create/drop namespaces.

ajantha-bhat avatar Apr 05 '23 17:04 ajantha-bhat

What's the status here?

snazy avatar Oct 13 '23 09:10 snazy

I didn't work on it.

@dimas-b, @adutra: Do you like to work on it? If not, I will check it next week.

ajantha-bhat avatar Oct 13 '23 10:10 ajantha-bhat

I can work on it 👍

adutra avatar Oct 13 '23 13:10 adutra