iceberg-python
iceberg-python copied to clipboard
feat: add support for multipart namespaces
Rationale for this change
This PR introduces support for multi-part namespaces, enabling more flexible and organized data layouts. This feature allows users to structure their tables within nested hierarchies (e.g., ('database', 'schema', 'table')), which is a common pattern in many data warehouse systems.
The level of support varies depending on the underlying catalog implementation. Here's a summary of the changes:
| Catalog Implementation | Multi-Part Namespace Support | Notes |
|---|---|---|
| REST Catalog | ✅ Yes | Fully supports multi-part namespaces as defined by the REST spec. |
| DynamoDB Catalog | ✅ Yes | The namespace is represented as a composite key. |
| File Catalog | ✅ Yes | Namespace parts are mapped to directory hierarchies in the file system. |
| In-Memory Catalog | ✅ Yes | Supports multi-part namespaces for testing purposes. |
| Hive Catalog | ❌ No | Spark and Hive Metastore do not support multi-part namespaces. |
| Glue Catalog | ❌ No | Uses AWS Glue databases which do not support multi-part namespaces. |
closes #2240