iceberg-rust icon indicating copy to clipboard operation
iceberg-rust copied to clipboard

Add `CatalogBuilder` trait.

Open liurenjie1024 opened this issue 8 months ago • 2 comments

liurenjie1024 avatar Apr 25 '25 02:04 liurenjie1024

For CatalogLoader trait, how about we have a CatalogConfig like:

pub struct CatalogConfig {
    name: String,
    uri: String,
    warehouse: String,
    props: HashMap<String, String>,
    ...others
}

So that our CatalogLoader just need to be like:

pub trait CatalogLoader {
    fn load(cfg: CatalogConfig) -> Result<Arc<dyn Catalog>>;
}

After this change, our loader itself is dyn compatible. Users who want to rest specific APIs can still use RestCatalogBuilder.

This change also makes it much easier for pyiceberg to connect since they can pass a CatalogConfig to rust directly without extra wrapper code.

What do you think?

Xuanwo avatar Apr 25 '25 03:04 Xuanwo

For CatalogLoader trait, how about we have a CatalogConfig like:

pub struct CatalogConfig { name: String, uri: String, warehouse: String, props: HashMap<String, String>, ...others } So that our CatalogLoader just need to be like:

pub trait CatalogLoader { fn load(cfg: CatalogConfig) -> Result<Arc<dyn Catalog>>; } After this change, our loader itself is dyn compatible. Users who want to rest specific APIs can still use RestCatalogBuilder.

This change also makes it much easier for pyiceberg to connect since they can pass a CatalogConfig to rust directly without extra wrapper code.

What do you think?

Sounds like a solution. Would you mind to create an example pr like https://github.com/apache/iceberg-rust/pull/1231 so that we have a better understand what it would look like?

liurenjie1024 avatar Apr 25 '25 03:04 liurenjie1024