incubator-xtable XTable REST Service for metadata and catalog sync.

Feature Request / Improvement

We have support to run XTable sync in continuous mode after this PR was merged https://github.com/apache/incubator-xtable/pull/593

There was interest in the community to build a REST service, users can call to trigger the metadata/catalog sync without the need for installing/packaging the dependencies in their project. Implementation details are still not finalized and feature owner can take it up.

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[x] I agree to follow this project's Code of Conduct

Mar 14 '25 23:03 vinishjail97

I am willing to take this up.

Mar 15 '25 14:03 vaibhavk1992

Hi team

Can we please add what all API end points we are planning to have in this and the consideration to take. I have listed a few though.

Trigger runsync Sync Endpoint: POST /api/v1/sync/runsync Description: Initiates the synchronization of metadata across supported table formats. Request Body: May include parameters specifying the scope or type of metadata to sync.

Endpoint: GET /api/v1/sync/status Description: Retrieves the current status of ongoing or recent synchronization processes. Query Parameters: Could include filters for specific sync operations or time ranges. List Supported Table Formats

Endpoint: GET /api/v1/formats Description: Returns a list of table formats supported by the synchronization service.

Mar 21 '25 18:03 vaibhavk1992

Will this PR be useful for the Polaris integration? I am curious to see the full list of APIs as well.

Mar 22 '25 23:03 sapienza88

@unical1988 Yes that's correct.

Mar 24 '25 15:03 vinishjail97

@vinishjail97 Have you checked this? https://aws.amazon.com/blogs/big-data/run-apache-xtable-in-aws-lambda-for-background-conversion-of-open-table-formats/

That should give you an endpoint to invoke to perform the conversion. You can do the same on Azure Functions.

Oct 02 '25 04:10 amoghnatu