scql

scql copied to clipboard

球球大佬们 拜托看看:::

修改配置数据源之后 p2p-tutorial-broker_alice-1 和 p2p-tutorial-broker_bob-1 起不来 设置为oceanbase了

启动： (cd examples/p2p-tutorial && docker compose up -d) 启动后查看： (base) [root@localhost scql]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c0c13a6b64d3 secretflow/scql:latest "/home/admin/bin/scq…" 44 seconds ago Up 40 seconds p2p-tutorial-engine_alice-1 8e0434e8dc89 secretflow/scql:latest "/home/admin/bin/bro…" 44 seconds ago Restarting (1) 5 seconds ago p2p-tutorial-broker_alice-1 01e4289f9ac5 oceanbase/oceanbase-ce "/bin/sh -c _boot" 44 seconds ago Up 40 seconds (health: starting) 2881/tcp p2p-tutorial-oceanbase-1 4af91d7b5bca secretflow/scql:latest "/home/admin/bin/bro…" 44 seconds ago Restarting (1) 5 seconds ago p2p-tutorial-broker_bob-1 0b357f01e3a2 secretflow/scql:latest "/home/admin/bin/scq…" 44 seconds ago Up 40 seconds p2p-tutorial-engine_bob-1

报错日志： (cd examples/p2p-tutorial && docker compose logs): broker_alice-1 | 2024-05-20 09:18:15.5209 INFO main.go:96 Starting to read config file: /home/admin/configs/config.yml broker_alice-1 | 2024-05-20 09:18:15.5209 INFO logger.go:147 /root/project/cmd/broker/main.go:218 broker_alice-1 | [error] failed to initialize database, got error dial tcp 127.0.0.1:2881: connect: connection refused broker_alice-1 | 2024-05-20 09:18:15.5209 FATAL main.go:142 Failed to create broker db: dial tcp 127.0.0.1:2881: connect: connection refused ... broker_bob-1 | 2024-05-20 09:18:15.5209 INFO main.go:96 Starting to read config file: /home/admin/configs/config.yml broker_bob-1 | 2024-05-20 09:18:15.5209 INFO logger.go:147 /root/project/cmd/broker/main.go:218 broker_bob-1 | [error] failed to initialize database, got error dial tcp 127.0.0.1:2881: connect: connection refused broker_bob-1 | 2024-05-20 09:18:15.5209 FATAL main.go:142 Failed to create broker db: dial tcp 127.0.0.1:2881: connect: connection refused

配置文件： examples/p2p-tutorial/broker/alice/conf/config.yml
—————————————————————————————————————— intra_server: protocol: http host: 0.0.0.0 port: 8080 inter_server: port: 8081 protocol: https cert_file: "/home/admin/tls/cert.crt" key_file: "/home/admin/tls/key.key" log_level: debug party_code: alice party_info_file: "/home/admin/configs/party_info.json" private_key_path: "/home/admin/configs/private_key.pem" intra_host: broker_alice:8080 engine: timeout: 120s protocol: https content_type: application/json uris: - for_peer: engine_alice:8003 for_self: engine_alice:8003 storage: type: mysql conn_str: "root:9y7lUPENo04Ja@tcp(127.0.0.1:2881)/brokeralice?charset=utf8mb4&parseTime=True&loc=Local&interpolateParams=true" max_idle_conns: 10 max_open_conns: 100 conn_max_idle_time: 2m conn_max_lifetime: 5m

examples/p2p-tutorial/broker/bob/conf/config.yml

---

intra_server: protocol: http host: 0.0.0.0 port: 8080 inter_server: port: 8081 protocol: https cert_file: "/home/admin/tls/cert.crt" key_file: "/home/admin/tls/key.key" log_level: debug party_code: bob party_info_file: "/home/admin/configs/party_info.json" private_key_path: "/home/admin/configs/private_key.pem" intra_host: broker_bob:8080 engine: timeout: 120s protocol: https content_type: application/json uris: - for_peer: engine_bob:8003 for_self: engine_bob:8003 storage: type: mysql conn_str: "root:9y7lUPENo04Ja@tcp(127.0.0.1:2881)/brokerbob?charset=utf8mb4&parseTime=True&loc=Local&interpolateParams=true" max_idle_conns: 10 max_open_conns: 100 conn_max_idle_time: 2m conn_max_lifetime: 5m

docker-compose.yml

---

version: '3.8' services: broker_alice: image: ${SCQL_IMAGE:-secretflow/scql:latest} command: - /home/admin/bin/broker - -config=/home/admin/configs/config.yml restart: always ports: - mode: host protocol: tcp published: ${ALICE_PORT:-8081} target: 8080 volumes: - ./broker/alice/conf/:/home/admin/configs/ - ./tls/root-ca.crt:/etc/ssl/certs/root-ca.crt - ./tls/broker_alice-ca.crt:/home/admin/tls/cert.crt - ./tls/broker_alice-ca.key:/home/admin/tls/key.key broker_bob: image: ${SCQL_IMAGE:-secretflow/scql:latest} command: - /home/admin/bin/broker - -config=/home/admin/configs/config.yml restart: always ports: - mode: host protocol: tcp published: ${BOB_PORT:-8082} target: 8080 volumes: - ./broker/bob/conf/:/home/admin/configs/ - ./tls/root-ca.crt:/etc/ssl/certs/root-ca.crt - ./tls/broker_bob-ca.crt:/home/admin/tls/cert.crt - ./tls/broker_bob-ca.key:/home/admin/tls/key.key engine_alice: cap_add: - NET_ADMIN command: - /home/admin/bin/scqlengine - --flagfile=/home/admin/engine/conf/gflags.conf image: ${SCQL_IMAGE:-secretflow/scql:latest} volumes: - ./engine/alice/conf/gflags.conf:/home/admin/engine/conf/gflags.conf - ./tls/engine_alice-ca.crt:/home/admin/engine/conf/cert.crt - ./tls/engine_alice-ca.key:/home/admin/engine/conf/key.key engine_bob: cap_add: - NET_ADMIN command: - /home/admin/bin/scqlengine - --flagfile=/home/admin/engine/conf/gflags.conf image: ${SCQL_IMAGE:-secretflow/scql:latest} volumes: - ./engine/bob/conf/gflags.conf:/home/admin/engine/conf/gflags.conf - ./tls/engine_alice-ca.crt:/home/admin/engine/conf/cert.crt - ./tls/engine_alice-ca.key:/home/admin/engine/conf/key.key oceanbase: image: oceanbase/oceanbase-ce environment: - MYSQL_ROOT_PASSWORD=9y7lUPENo04Ja - TZ=Asia/Shanghai healthcheck: retries: 10 test: - CMD - mysqladmin - ping - -h - mysql timeout: 20s expose: - "2881" restart: always volumes: - ./oceanbase/initdb:/docker-entrypoint-initdb.d

oceanbase需要配合oceanbase proxy使用， scql engine没法直接访问oceanbase。

oceanbase需要配合oceanbase proxy使用， scql engine没法直接访问oceanbase。

感谢，已经解决~

你好 请教一下 设置数据源为kingbase 如果使用kingbase（postgresql mode）需要配合什么使用 才能正常启动 ？ 报错如下： (base) [root@localhost scql]# (cd examples/p2p-tutorial && docker compose logs) WARN[0000] /root/secretflow/secretscql/scql/examples/p2p-tutorial/docker-compose.yml: version is obsolete broker_alice-1 | 2024-05-30 09:47:38.5309 INFO main.go:96 Starting to read config file: /home/admin/configs/config.yml broker_alice-1 | [mysql] 2024/05/30 09:48:38 packets.go:37: unexpected EOF broker_alice-1 | [mysql] 2024/05/30 09:49:38 packets.go:37: unexpected EOF broker_alice-1 | [mysql] 2024/05/30 09:50:38 packets.go:37: unexpected EOF broker_alice-1 | 2024-05-30 09:50:38.5309 INFO logger.go:147 /root/project/cmd/broker/main.go:218 broker_alice-1 | [error] failed to initialize database, got error driver: bad connection broker_alice-1 | 2024-05-30 09:50:38.5309 FATAL main.go:142 Failed to create broker db: driver: bad connection broker_alice-1 | 2024-05-30 09:50:39.5309 INFO main.go:96 Starting to read config file: /home/admin/configs/config.yml broker_alice-1 | [mysql] 2024/05/30 09:51:39 packets.go:37: unexpected EOF engine_bob-1 | 2024-05-30 09:47:37.827 [info] [main.cc:main:419] [sciengine] Adding MuxReceiverService into brpc server engine_bob-1 | 2024-05-30 09:47:37.831 [info] [main.cc:BuildRouter:216] [sciengine] Building EmbedRouter from json conf engine_bob-1 | 2024-05-30 09:47:37.843 [error] [main.cc:main:437] [sciengine] Fail to build engine service, msg=[engine/datasource/embed_router.cc:77] Fail EmbedRouterConf, reason=INVALID_ARGUMENT:(datasources[0].kind): invalid value "PostgreSQL" for type type.googleapis.com/scql.engine.DataSourceKind engine_alice-1 | 2024-05-30 09:47:37.932 [info] [main.cc:main:419] [sciengine] Adding MuxReceiverService into brpc server engine_alice-1 | 2024-05-30 09:47:37.945 [info] [main.cc:BuildRouter:216] [sciengine] Building EmbedRouter from json conf engine_alice-1 | 2024-05-30 09:47:37.963 [error] [main.cc:main:437] [sciengine] Fail to build engine service, msg=[engine/datasource/embed_router.cc:77] Fail EmbedRouterConf, reason=INVALID_ARGUMENT:(datasources[0].kind): invalid value "PostgreSQL" for type type.googleapis.com/scql.engine.DataSourceKind broker_bob-1 | 2024-05-30 09:47:38.5309 INFO main.go:96 Starting to read config file: /home/admin/configs/config.yml broker_bob-1 | [mysql] 2024/05/30 09:48:38 packets.go:37: unexpected EOF broker_bob-1 | [mysql] 2024/05/30 09:49:38 packets.go:37: unexpected EOF broker_bob-1 | [mysql] 2024/05/30 09:50:38 packets.go:37: unexpected EOF broker_bob-1 | 2024-05-30 09:50:38.5309 INFO logger.go:147 /root/project/cmd/broker/main.go:218 broker_bob-1 | [error] failed to initialize database, got error driver: bad connection broker_bob-1 | 2024-05-30 09:50:38.5309 FATAL main.go:142 Failed to create broker db: driver: bad connection broker_bob-1 | 2024-05-30 09:50:39.5309 INFO main.go:96 Starting to read config file: /home/admin/configs/config.yml broker_bob-1 | [mysql] 2024/05/30 09:51:39 packets.go:37: unexpected EOF

对于engine的datasource配置，使用全大写的POSTGRESQL试下。 对于broker，暂时只支持mysql/sqlite，不支持postresql。

对于engine的datasource配置，使用全大写的POSTGRESQL试下。 对于broker，暂时只支持mysql/sqlite，不支持postresql。

感谢，有用！

你好，如果使用 达梦数据库 作为数据源 应该选择MySQL/SQLite3/PostgreSQL/CSVDB/ArrowSQ中的什么类型呢？

您好，我们之前没测试验证过达梦数据库，建议您核对其接入api确认是否能接入到SCQL。 BTW：CSVDB/ARROWSQL应该可以先排除，有环境的话，其它的不妨先尝试下。

好的，到时候尝试后再反馈，感谢~

对于engine的datasource配置，使用全大写的POSTGRESQL试下。 对于broker，暂时只支持mysql/sqlite，不支持postresql。

感谢，有用！

你好，请问一下你们是否已成功设置数据源为kingbase？如果成功，用的是p2p模式还是中心化模式？因为上述提到对于broker，暂时只支持mysql/sqlite，不支持postresql，是否认为不能在p2p模式中实现kingbase的数据源？ 劳烦，期待您的回复！