TiBigData
TiBigData copied to clipboard
tidb-incubator
Unable to start Trino after using PrestoSQL-TiDB-Connector
I followed this doc to build the connector and the build finished successfully. https://github.com/tidb-incubator/TiBigData/tree/master/prestosql
my tidb plugin folder content as blew:
root@trino-worker01:/home/trino/trino-server-361/etc/catalog# ls -l /home/trino/trino-server-361/plugin/tidb/
total 38940
-rw-r--r-- 1 root root 4467 9月 28 15:03 aopalliance-1.0.jar
-rw-r--r-- 1 root root 44329 9月 28 15:03 bigdata-core-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root 26353 9月 28 15:03 bootstrap-202.jar
-rw-r--r-- 1 root root 805301 9月 28 15:03 bval-jsr-2.0.0.jar
-rw-r--r-- 1 root root 415348 9月 28 15:03 cglib-nodep-3.3.0.jar
-rw-r--r-- 1 root root 230905 9月 28 15:03 checker-qual-3.8.0.jar
-rw-r--r-- 1 root root 83762 9月 28 15:03 configuration-202.jar
-rw-r--r-- 1 root root 13854 9月 28 15:03 error_prone_annotations-2.5.1.jar
-rw-r--r-- 1 root root 4617 9月 28 15:03 failureaccess-1.0.1.jar
-rw-r--r-- 1 root root 2874025 9月 28 15:03 guava-30.1.1-jre.jar
-rw-r--r-- 1 root root 856934 9月 28 15:03 guice-4.2.3.jar
-rw-r--r-- 1 root root 156005 9月 28 15:03 HikariCP-3.4.5.jar
-rw-r--r-- 1 root root 8781 9月 28 15:03 j2objc-annotations-1.3.jar
-rw-r--r-- 1 root root 68167 9月 28 15:03 jackson-annotations-2.11.1.jar
-rw-r--r-- 1 root root 349108 9月 28 15:03 jackson-core-2.10.3.jar
-rw-r--r-- 1 root root 1404171 9月 28 15:03 jackson-databind-2.10.3.jar
-rw-r--r-- 1 root root 83893 9月 28 15:03 jackson-datatype-guava-2.10.3.jar
-rw-r--r-- 1 root root 34402 9月 28 15:03 jackson-datatype-jdk8-2.10.3.jar
-rw-r--r-- 1 root root 72117 9月 28 15:03 jackson-datatype-joda-2.10.3.jar
-rw-r--r-- 1 root root 105898 9月 28 15:03 jackson-datatype-jsr310-2.10.3.jar
-rw-r--r-- 1 root root 9328 9月 28 15:03 jackson-module-parameter-names-2.10.3.jar
-rw-r--r-- 1 root root 26586 9月 28 15:03 javax.annotation-api-1.3.2.jar
-rw-r--r-- 1 root root 2497 9月 28 15:03 javax.inject-1.jar
-rw-r--r-- 1 root root 16537 9月 28 15:03 jcl-over-slf4j-1.7.30.jar
-rw-r--r-- 1 root root 2920901 9月 28 15:03 jmxutils-1.21.jar
-rw-r--r-- 1 root root 32601 9月 28 15:03 json-202.jar
-rw-r--r-- 1 root root 19936 9月 28 15:03 jsr305-3.0.2.jar
-rw-r--r-- 1 root root 2199 9月 28 15:03 listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
-rw-r--r-- 1 root root 3908 9月 28 15:03 log-202.jar
-rw-r--r-- 1 root root 23702 9月 28 15:03 log4j-over-slf4j-1.7.30.jar
-rw-r--r-- 1 root root 471901 9月 28 15:03 logback-core-1.2.3.jar
-rw-r--r-- 1 root root 19470 9月 28 15:03 log-manager-202.jar
-rw-r--r-- 1 root root 1006956 9月 28 15:45 mysql-connector-java-5.1.48.jar
-rw-r--r-- 1 root root 48203 9月 28 15:03 prestosql-connector-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root 41203 9月 28 15:03 slf4j-api-1.7.25.jar
-rw-r--r-- 1 root root 8510 9月 28 15:03 slf4j-jdk14-1.7.30.jar
-rw-r--r-- 1 root root 103273 9月 28 15:03 tidb-jdbc-0.0.5-SNAPSHOT.jar
-rw-r--r-- 1 root root 27276468 9月 28 15:03 tikv-client-java-3.2.0-SNAPSHOT.jar
-rw-r--r-- 1 root root 19555 9月 28 15:03 units-1.6.jar
-rw-r--r-- 1 root root 93107 9月 28 15:03 validation-api-2.0.1.Final.jar
And I add the catalog for tidb, content as blow.
connector.name=tidb
tidb.database.url=jdbc:mysql://xxx:3306/xxx
tidb.username=xxx
tidb.password=xxx
tidb.filter-push-down=true
But after I restart the trino, it errored as below.
2021-09-28T15:49:51.601+0800 ERROR main io.trino.server.Server No service providers of type io.trino.spi.Plugin
java.lang.IllegalStateException: No service providers of type io.trino.spi.Plugin
at com.google.common.base.Preconditions.checkState(Preconditions.java:591)
at io.trino.server.PluginManager.loadPlugin(PluginManager.java:139)
at io.trino.server.PluginManager.loadPlugin(PluginManager.java:129)
at io.trino.server.ServerPluginsProvider.loadPlugins(ServerPluginsProvider.java:48)
at io.trino.server.PluginManager.loadPlugins(PluginManager.java:110)
at io.trino.server.Server.doStart(Server.java:122)
at io.trino.server.Server.lambda$start$0(Server.java:77)
at io.trino.$gen.Trino_361____20210928_074943_1.run(Unknown Source)
at io.trino.server.Server.start(Server.java:77)
at io.trino.server.TrinoServer.main(TrinoServer.java:38)
My trino version: 361
mysql connector I use: mysql-connector-java-5.1.48.jar
Please help if anything wrong with my setup?
I manage to fixe the above issue by change prestosql to trino in build process. Now I am encountering below issue when start the trino server.
error when adding tidb.filter-push-down=true in the catalog file
If I add tidb.filter-push-down=true in the catalog file, below error will occurr.
2021-09-28T17:11:03.612+0800 ERROR main io.trino.server.Server Configuration errors:
1) Error: Configuration property 'tidb.filter-push-down' was not used
1 error
io.airlift.bootstrap.ApplicationConfigurationException: Configuration errors:
1) Error: Configuration property 'tidb.filter-push-down' was not used
1 error
at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:233)
at io.tidb.bigdata.trino.tidb.TiDBConnectorFactory.create(TiDBConnectorFactory.java:57)
at io.trino.connector.ConnectorManager.createConnector(ConnectorManager.java:360)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:215)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:207)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:193)
at io.trino.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
at io.trino.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
at io.trino.server.Server.doStart(Server.java:124)
at io.trino.server.Server.lambda$start$0(Server.java:77)
at io.trino.$gen.Trino_361____20210928_091053_1.run(Unknown Source)
at io.trino.server.Server.start(Server.java:77)
at io.trino.server.TrinoServer.main(TrinoServer.java:38)
2021-09-28T17:11:03.614+0800 INFO Thread-81 io.airlift.bootstrap.LifeCycleManager JVM is shutting down, cleaning up
2021-09-28T17:11:03.614+0800 INFO Thread-81 io.airlift.bootstrap.LifeCycleManager Life cycle stopping...
2021-09-28T17:11:03.615+0800 INFO Thread-81 io.airlift.bootstrap.LifeCycleManager Life cycle stopped
error WITHOUT adding tidb.filter-push-down=true in the catalog file
If remove tidb.filter-push-down=true, then server can be started successfully, but when I run query in cli, Internal error will occurred.
trino:skyline_test> select * from test1;
Query 20210928_101552_00008_hcyiy failed: Internal error
In the stack trace log it shows as below:
java.lang.UnsupportedOperationException: undefined
at io.trino.spi.connector.ConnectorSplitManager.getSplits(ConnectorSplitManager.java:35)
at io.trino.split.SplitManager.getSplits(SplitManager.java:86)
at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitScanAndFilter(DistributedExecutionPlanner.java:206)
at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:188)
at io.trino.sql.planner.DistributedExecutionPlanner$Visitor.visitTableScan(DistributedExecutionPlanner.java:159)
at io.trino.sql.planner.plan.TableScanNode.accept(TableScanNode.java:222)
at io.trino.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:127)
at io.trino.sql.planner.DistributedExecutionPlanner.doPlan(DistributedExecutionPlanner.java:134)
at io.trino.sql.planner.DistributedExecutionPlanner.plan(DistributedExecutionPlanner.java:104)
at io.trino.execution.SqlQueryExecution.planDistribution(SqlQueryExecution.java:508)
at io.trino.execution.SqlQueryExecution.start(SqlQueryExecution.java:415)
at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:237)
at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
at io.trino.$gen.Trino_361____20210928_100845_2.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
tidb.filter-push-down
Hi @myheartsgoon, thanks for reporting issue. Could you please try running the same query with trino version 359?
tidb.filter-push-down
Hi @myheartsgoon, thanks for reporting issue. Could you please try running the same query with trino version 359?
Thanks for your prompt response, I just tested with version 359, when not specifying tidb.filter-push-down=true, it is working as expected. (I guess it is due to this change since 360? https://trino.io/docs/current/release/release-360.html#spi)
When I adding tidb.filter-push-down=true in catalog file, it is still not working, same error message as I describe above.
2021-09-29T11:47:43.085+0800 ERROR main io.trino.server.Server Configuration errors:
1) Error: Configuration property 'tidb.filter-push-down' was not used
1 error
io.airlift.bootstrap.ApplicationConfigurationException: Configuration errors:
1) Error: Configuration property 'tidb.filter-push-down' was not used
1 error
at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:233)
at io.tidb.bigdata.trino.tidb.TiDBConnectorFactory.create(TiDBConnectorFactory.java:57)
at io.trino.connector.ConnectorManager.createConnector(ConnectorManager.java:360)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:215)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:207)
at io.trino.connector.ConnectorManager.createCatalog(ConnectorManager.java:193)
at io.trino.metadata.StaticCatalogStore.loadCatalog(StaticCatalogStore.java:88)
at io.trino.metadata.StaticCatalogStore.loadCatalogs(StaticCatalogStore.java:68)
at io.trino.server.Server.doStart(Server.java:124)
at io.trino.server.Server.lambda$start$0(Server.java:77)
at io.trino.$gen.Trino_359____20210929_034733_1.run(Unknown Source)
at io.trino.server.Server.start(Server.java:77)
at io.trino.server.TrinoServer.main(TrinoServer.java:38)
But for now I am good as at least I can query tidb data with presto, although without filter pushdown feature.
Thanks for your prompt response, I just tested with version 359, when not specifying
tidb.filter-push-down=true, it is working as expected. (I guess it is due to this change since 360? https://trino.io/docs/current/release/release-360.html#spi)
Trino / PrestoSQL / PrestoDB are evolving pretty fast, it's usually required to match version used to compile and run connector.
@humengyu2012 Can you take a look at configuration not used error?
Presto will always enable filter-push-down and can not be disabled, so you don't need to configure it.
