incubator-xtable icon indicating copy to clipboard operation
incubator-xtable copied to clipboard

Add Client Extensibility via ServiceLoader

Open lmccay opened this issue 7 months ago • 4 comments

Discussion within issue #13 has moved farther away from the original intent of that issue.

This issue represents the ability to extend the Source and Target Clients by using ServiceLoader to find all of the relevant clients on the classpath and filter to the name provided through the TargetClient.getTableFormatName(). This allows for being able to specify the name rather than an enum.

In the end, one would be able to extend onetable with another client by implementing the TargetClient, adding a ServiceLoader config file in a file in the jar at resources/META-INF/services/io.onetable.spi.sync.TargetClient with the client fully qualified classname

Well written article on the use of ServiceLoader and the target usecase which I think aligns well with what we have been discussing. https://pedrorijo.com/blog/java-service-loader/

There is also a pointer to a google project to automate the service config file which looks really nice. I wasn't aware of that before.

example:

##########################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
##########################################################################
io.onetable.delta.DeltaClient
io.onetable.hudi.HudiTargetClient
io.onetable.iceberg.IcebergClient

The above illustrates the builtin clients.

A custom extension would like something like:

##########################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
##########################################################################
my.super.custom.TableClient

This change requires some refactoring and newly created interfaces for some of the implementation classes.

lmccay avatar Dec 16 '23 04:12 lmccay