trino
trino copied to clipboard
trinodb
Add StarRocks connector
Description
Fix https://github.com/trinodb/trino/issues/17329
Additional context and related issues
Release notes
( ) This is not user-visible or docs only and no release notes are required. ( ) Release notes are required, please propose a release note for me. (x) Release notes are required, with the following suggested text:
# StarRocks
* Add StarRocks connector. ({issue}`17329`)
@Praveen2112 Thank you so much for the review! I will mark the PR as the draft I don't have time now, I will open this PR after the 9th, Sorry about the mass.
2023-09-29T15:52:32.981Z WARN main io.trino.tests.product.launcher.env.Environment +++++++++Container starrocks++++, id: 8f64dc15b505d332f9e15b37299a3c9f24ee62b25c94349b223f0f2afacb8966, start up time:26.02s, healthy: false
2023-09-29T15:52:32.996Z WARN main io.trino.tests.product.launcher.env.Environment Could not start environment 'multinode-star-rocks'
java.lang.RuntimeException: com.github.dockerjava.api.exception.NotFoundException: Status 404: {"message":"No such container: 8f64dc15b505d332f9e15b37299a3c9f24ee62b25c94349b223f0f2afacb8966"}
at io.trino.tests.product.launcher.env.Environment.tryStart(Environment.java:177)
at dev.failsafe.Functions.lambda$toCtxSupplier$11(Functions.java:243)
at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
at dev.failsafe.internal.RetryPolicyExecutor.lambda$apply$0(RetryPolicyExecutor.java:74)
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
at dev.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:112)
at io.trino.tests.product.launcher.env.Environment.start(Environment.java:130)
at io.trino.tests.product.launcher.cli.TestRun$Execution.startEnvironment(TestRun.java:310)
at io.trino.tests.product.launcher.cli.TestRun$Execution.tryExecuteTests(TestRun.java:255)
at dev.failsafe.Functions.lambda$toCtxSupplier$11(Functions.java:243)
at dev.failsafe.Functions.lambda$get$0(Functions.java:46)
at dev.failsafe.internal.TimeoutExecutor.lambda$apply$1(TimeoutExecutor.java:93)
at dev.failsafe.SyncExecutionImpl.executeSync(SyncExecutionImpl.java:187)
at dev.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:376)
at dev.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:112)
at io.trino.tests.product.launcher.cli.TestRun$Execution.call(TestRun.java:233)
at io.trino.tests.product.launcher.cli.SuiteRun$Execution.runTest(SuiteRun.java:303)
at io.trino.tests.product.launcher.cli.SuiteRun$Execution.executeSuiteTestRun(SuiteRun.java:281)
at io.trino.tests.product.launcher.cli.SuiteRun$Execution.lambda$runSuites$2(SuiteRun.java:212)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Collections$2.tryAdvance(Collections.java:4853)
at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4861)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at io.trino.tests.product.launcher.cli.SuiteRun$Execution.lambda$runSuites$3(SuiteRun.java:213)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at io.trino.tests.product.launcher.cli.SuiteRun$Execution.runSuites(SuiteRun.java:199)
at io.trino.tests.product.launcher.cli.SuiteRun$Execution.call(SuiteRun.java:185)
at io.trino.tests.product.launcher.cli.SuiteRun$Execution.call(SuiteRun.java:136)
at io.trino.tests.product.launcher.cli.LauncherCommand.call(LauncherCommand.java:61)
at io.trino.tests.product.launcher.cli.LauncherCommand.call(LauncherCommand.java:29)
at picocli.CommandLine.executeUserObject(CommandLine.java:2041)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
at picocli.CommandLine.execute(CommandLine.java:2170)
at io.trino.tests.product.launcher.cli.Launcher.execute(Launcher.java:77)
at io.trino.tests.product.launcher.cli.Launcher.main(Launcher.java:66)
Caused by: com.github.dockerjava.api.exception.NotFoundException: Status 404: {"message":"No such container: 8f64dc15b505d332f9e15b37299a3c9f24ee62b25c94349b223f0f2afacb8966"}
at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:241)
at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:202)
at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.get(DefaultInvocationBuilder.java:74)
at org.testcontainers.shaded.com.github.dockerjava.core.exec.InspectContainerCmdExec.execute(InspectContainerCmdExec.java:31)
at org.testcontainers.shaded.com.github.dockerjava.core.exec.InspectContainerCmdExec.execute(InspectContainerCmdExec.java:13)
at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
at org.testcontainers.shaded.com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:33)
at org.testcontainers.shaded.com.github.dockerjava.core.command.InspectContainerCmdImpl.exec(InspectContainerCmdImpl.java:51)
at org.testcontainers.containers.ContainerState.getCurrentContainerInfo(ContainerState.java:129)
at io.trino.tests.product.launcher.env.Environment.containerIsHealthy(Environment.java:338)
at java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
at java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:1002)
at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
at java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.allMatch(ReferencePipeline.java:637)
at io.trino.tests.product.launcher.env.Environment.allContainersHealthy(Environment.java:327)
at io.trino.tests.product.launcher.env.Environment.tryStart(Environment.java:167)
... 44 more
cc @wendigo Do you know why encounter this error? The Environment.containerIsHealthy returns false for the StarRocks container, so I log the error, it shows the container Id exists, but fail on getCurrentContainerInfo due to above error
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
:wave: @chenjian2664 - this PR has become inactive. If you're still interested in working on it, please let us know.
We're working on closing out old and inactive PRs, so if you're too busy or this has too many merge conflicts to be worth picking back up, we'll be making another pass to close it out in a few weeks.
@mosabua Yes. I would greatly appreciate it if you/or someone could take a moment to review this pr
@chenjian2664 could you rebase and ensure all prior comments are addressed. Then I hope I can get some help from @Praveen2112 and others
Thanks for the rebase @chenjian2664 .. hopefully @Praveen2112 and @hashhar can take a look soon. And CI passes...
@mosabua Sure, Thanks for your time! @hashhar @Praveen2112 It would be wonderful to read your thoughts and suggestions!
Looks like there are some failures .. can you please ensure that the build works locally on your workstation and push any further necessary changes.
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
@chenjian2664,@mosabua:- Is this connector still on ? The native MySQL connector does is not fully compatible with Starrocks (certain commands and performance). Would be great if this official SR connector can be added.
Thanks!
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.
@chenjian2664 - Is this connector still on ? The native MySQL connector does is not fully compatible with Starrocks (certain commands and performance). Would be great if this official SR connector can be added.
@hashhar seeing that you skimmed this before, are you interested in giving this a review? https://github.com/trinodb/trino/pull/17330#pullrequestreview-1679365431
I just gave the docs a heavy comb and looked through the testing and it covered all the bases IMO.
We are interested to get this reviewed, merged and released. However, currently there have not been sufficient reviews and testing feedback from community members and especially also maintainers.
If anyone can help testing, especially people with more knowledge about StarRocks, that would be helpful
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
We are interested to get this reviewed, merged and released. However, currently there have not been sufficient reviews and testing feedback from community members and especially also maintainers.
If anyone can help testing, especially people with more knowledge about StarRocks, that would be helpful
We have back ported relevant source codes and used it in our own Trino 423 environment(with StarRocks 3.2.2) based on this project: https://github.com/footprintanalytics/trino-starrrocks/tree/starrocks_connector
This project is also originated from the PR now.
It seems work, so personally i think that it can be merged, after the jdk 23 ut error is fixed.
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.
I am adding the stale-ignore label since this is still a valid PR and useful connector to potentially add.
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua