Nok Lam Chan

^ To be more specific, I was referring mainly to the python binding, i.e. PyIceberg and rust-delta(python). Iceberg itself is fairly mature, especially with the catalog etc, but the python...

Any chance I can take this ticket or work together on this? I have explored this a little bit a while ago and would be a great opportunities to continue...

@ElenaKhaustova I have left some questions at the end since it's not a PR yet. https://noklam.github.io/blog/posts/pyiceberg/2024-11-18-PyIcebergDataset.html ```md # Questions - What does it means when we said " if we...

> It's clear we can easily version artifacts (tabular data), but what about versioning catalogs/projects—more high-level entities and non-tabular data? - Iceberg (table) was designed for tabular data, so non-tabular...

I put together an example here: https://github.com/noklam/kedro-example/tree/master/iceberg-incremental TL;DR Summary - Iceberg support Table only, so won't be a full replacement of Kedro own filebased versioning. - Versioning is fairly flexible...

> Yes but many users (including ourselves! like @DimedS's ETL for PyPI data) are using Kedro for data pipelines that require UPSERT/merge operations, see long discussion and pointers to user...

https://aws.amazon.com/s3/features/tables/ This may changes the conclusion a bit, S3 Table now support Iceberg natively. This may take away some need for Catalog / cleaning work that are often done by...

TL;DR **I agree we should keep one of the two tickets only.** > This can be solved if https://github.com/kedro-org/kedro/issues/3932 is done, but the last is a much more complex task,...

@astrojuanlu TL;DR, I think we should close one of the ticket. I don't want to get into the solution yet since I think the scope is not clear enough yet....

Run into the same issue. Would the team open to PR to make the documentations host on ReadTheDocs so that it's versioned?