nessie
nessie copied to clipboard
Creating/dropping the namespace doesn't capture committer and author information.
Commit log entry: (doesn't contain author and committer info)
[LogEntry{commitMeta=CommitMeta{hash=06f3272a0448bcc177f80313d19b1315973e918637b3bff6c3262b588a25bedf, committer=, author=, allAuthors=[], signedOffBy=null, allSignedOffBy=[], message=delete namespace ns, commitTime=2023-04-05T06:29:58.785Z, authorTime=2023-04-05T06:29:58.785Z, allProperties={}, parentCommitHashes=[]}, additionalParents=[], parentCommitHash=b0911d20316931f42ce9503118f78164771fa00e7edbef4a5f4af5e65bf8ddb0, operations=null},
Should these commits happen from NessieIcebergClient
or pass the committer and author info to the server?
I am in favour of committing from NessieIcebergClient
to have uniform behaviour with other commits.
https://github.com/projectnessie/nessie/blob/ad6d360f5fa57f99e998667d9436fddaf3b11cf0/api/client/src/main/java/org/projectnessie/client/util/v2api/ClientSideCreateNamespace.java#L89-L94
cc: @dimas-b, @snazy
Normally, committer
is derived from AuthN data. Clients cannot set it.
IIRC, the Iceberg-side Nessie code should already be setting author
data... I may be mistaken, though. @ajantha-bhat : Could you double check?
ClientSideCreateNamespace
is more of a backward compatibility fixture to simplify migration from API v1 to v2, IMHO.
Ideally, API v2 clients should use the commitMultipleOperations()
method directly in order to create namespaces, and set proper author
data.
I have mentioned the code snippet above where we are not setting the commit meta with author and other information.
For a normal commit like create table, Iceberg side we add commit meta like this
https://github.com/apache/iceberg/blob/d485cc83ec58bd587fbfaaacebcf624c7f4c44d3/nessie/src/main/java/org/apache/iceberg/nessie/NessieUtil.java#L65-L91
Yes, I edited the above comment a bit after posting it :)
yes. My question is commitMultipleOperations()
is called at client side for create table, commit table, etc.
So, should we also have commitMultipleOperations()
to create a namespace at the client side? So, it will also handle the author and committer info.
Re: the java client API, it might be worth adding an input method for CommitMeta
to CreateNamespaceBuilder
and similar builder. The new method can be supported only in API v2 (ignored in v1).
So, should we also have commitMultipleOperations() to create a namespace at the client side? So, it will also handle the author and committer info.
This is preferable, I believe. REST API v2 does not have a dedicated endpoint for namespace manipulation. All changes should be done via the commit API, so the less wrappers we have around that, the better, IMHO.
Ack, I will take a look at adding commitMultipleOperations()
at the client side and cleaning up the code for create/drop namespaces.
What's the status here?
I didn't work on it.
@dimas-b, @adutra: Do you like to work on it? If not, I will check it next week.
I can work on it 👍