CSGHub 【BUG】组织数据集无法相互访问

问题描述

部署方式使用：docker compose -f all-in-one.yml up -d

组织中的任意一个用户创建数据集（所有者为组织），组织内其他用户不可见
点击组织首页中，创建组织的用户同样看不到任何数据集
设置public后所有用户均可见（包含未登录用户），失去组织的约束

Jun 25 '24 09:06 ioo0s

复现步骤

创建用户组
创建用户组数据集
成功创建数据集，并且能够查看
再次点击组织，发现数据集位置未显示刚刚创建的数据集

猜测问题

注册的个人用户创建数据集
创建个人数据集
主页中可以查看到该数据集，证明创建成功
个人主页无法查看数据集

Jun 25 '24 09:06 ioo0s

感谢提供这么详细的信息！

个人页也看不到自己创建的数据集，可以判断跟组织的权限无关。这里可能的原因是个人页获取数据失败了。

@ioo0s 能否在刷新个人主页的同时查看一下服务端的日志，看看请求是否到达服务端，是否有错误？

docker compose logs -f --since 1m starhub_server

服务端会记录类似这样的访问日志

starhub-starhub_server-1  | {"time":"2024-06-26T03:39:55.618968801Z","level":"INFO","msg":"http request","ip":"219.142.137.156","method":"GET","latency(ms)":10,"status":200,"current_user":"test","auth_type":"JWT","url":"/api/v1/user/test/datasets?per=6&page=1"}

Jun 26 '24 03:06 Rader

可以的首先我使用咱们的all_in_one部署的服务没有starthub服务，对应的是csghub服务

csghub_portal-1  | Started GET "/profile/ios" for ip at 2024-06-26 03:58:59 +0000
csghub_portal-1  | Cannot render console from ip! Allowed networks: 127.0.0.0/127.255.255.255, ::1
csghub_portal-1  | Processing by ProfileController#index as HTML
csghub_portal-1  |   Parameters: {"user_id"=>"ios"}
csghub_portal-1  |   User Load (0.3ms)  SELECT "users".* FROM "users" WHERE "users"."login_identity" = $1 LIMIT $2  [["login_identity", "xxxxxxxx"], ["LIMIT", 1]]
csghub_portal-1  |   ↳ app/helpers/sessions_helper.rb:11:in `current_user'
csghub_portal-1  |   User Load (0.4ms)  SELECT "users".* FROM "users" WHERE "users"."name" = $1 LIMIT $2  [["name", "ios"], ["LIMIT", 1]]
csghub_portal-1  |   ↳ app/controllers/profile_controller.rb:5:in `index'
csghub_portal-1  |   Rendering layout layouts/application.html.erb
csghub_portal-1  |   Rendering profile/index.html.erb within layouts/application
csghub_portal-1  |   SystemConfig Load (0.4ms)  SELECT "system_configs".* FROM "system_configs" ORDER BY "system_configs"."id" ASC LIMIT $1  [["LIMIT", 1]]
csghub_portal-1  |   ↳ app/services/aws_s3.rb:73:in `load_s3_config'
csghub_portal-1  |   Organization Load (0.6ms)  SELECT "organizations".* FROM "organizations" INNER JOIN "org_memberships" ON "organizations"."id" = "org_memberships"."organization_id" WHERE "org_memberships"."user_id" = $1  [["user_id", 2]]
csghub_portal-1  |   ↳ app/views/profile/index.html.erb:11
csghub_portal-1  |   Rendered profile/index.html.erb within layouts/application (Duration: 5.0ms | Allocations: 2517)
csghub_portal-1  |   CACHE User Load (0.0ms)  SELECT "users".* FROM "users" WHERE "users"."login_identity" = $1 LIMIT $2  [["login_identity", "xxxxx"], ["LIMIT", 1]]
csghub_portal-1  |   ↳ app/helpers/sessions_helper.rb:11:in `current_user'
csghub_portal-1  |   Rendered partials/_navbar.html.erb (Duration: 0.8ms | Allocations: 512)
csghub_portal-1  |   Rendered partials/_footer.html.erb (Duration: 0.0ms | Allocations: 8)
csghub_portal-1  |   Rendered layout layouts/application.html.erb (Duration: 6.9ms | Allocations: 3902)
csghub_portal-1  | Completed 200 OK in 12ms (Views: 6.3ms | ActiveRecord: 1.6ms | Allocations: 5604)
csghub_portal-1  | 
csghub_portal-1  | 
gitea-1          | 2024/06/26 03:58:59 ...c/net/http/server.go:2136:ServeHTTP() [I] PING DATABASE postgres
gitea-1          | 2024/06/26 03:58:59 ...eb/routing/logger.go:102:func1() [I] router: completed GET /api/healthz for 127.0.0.1:39668, 200 OK in 0.5ms @ healthcheck/check.go:67(healthcheck.Check)
csghub_server-1  | {"time":"2024-06-26T03:59:02.942153775Z","level":"ERROR","msg":"refresh status all failed","error":"Get \"http://localhost:8082/status-all\": dial tcp [::1]:8082: connect: connection refused"}

Jun 26 '24 04:06 ioo0s

测试了你们最新的部署方式https://github.com/OpenCSGs/CSGHub/tree/main/script/all_in_one 同样存在该问题，希望能尽快修复！！！

Jun 27 '24 02:06 ioo0s

csghub_server-1 | {"time":"2024-06-26T03:59:02.942153775Z","level":"ERROR","msg":"refresh status all failed","error":"Get "http://localhost:8082/status-all": dial tcp [::1]:8082: connect: connection refused"}

@ioo0s 个人主页刷新完成之后，csghub_server-1 的日志就只有这一条吗？

Jun 27 '24 09:06 Rader

是的，没有其他多余的log

Jun 28 '24 00:06 ioo0s

Got the same error when I just compose up, csghub_server will endless restart. Test on https://raw.githubusercontent.com/OpenCSGs/csghub/main/all-in-one.yml and https://opencsg-public-resource.oss-cn-beijing.aliyuncs.com/csghub/all-in-one-CN.yml, the same error.

csghub_server-1  | Database setup...
csghub_server-1  | Migration init
csghub_server-1  | init logger, level: INFO, format: json
csghub_server-1  | Migration migrate
csghub_server-1  | init logger, level: INFO, format: json
csghub_server-1  | {"time":"2024-07-08T08:47:53.771818081Z","level":"INFO","msg":"there are no new migrations to run (database is up to date)"}
csghub_server-1  | Start server...
csghub_server-1  | init logger, level: INFO, format: json
csghub_server-1  | {"time":"2024-07-08T08:47:53.796174633Z","level":"INFO","msg":"FIFOScheduler run started"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.796372517Z","level":"ERROR","msg":"refresh status all failed","error":"Get \"http://localhost:8082/status-all\": dial tcp [::1]:8082: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808899484Z","level":"ERROR","msg":"Error sending request:","!BADKEY":"Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808925276Z","level":"ERROR","msg":"Fail to create mirror server access token","error: ":"Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808928244Z","level":"ERROR","msg":"Failed to find or create token","error: ":"Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808936128Z","level":"ERROR","msg":"fail to create git mirror server,error:Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | Error: failed to init router: error creating user controller:failed to create space component,error:fail to create git mirror server,error:Post "http://localhost:3001/api/v1/users/root/tokens": dial tcp [::1]:3001: connect: connection refused

Jul 08 '24 08:07 vagetablechicken

Got the same error when I just compose up, csghub_server will endless restart. Test on https://raw.githubusercontent.com/OpenCSGs/csghub/main/all-in-one.yml and https://opencsg-public-resource.oss-cn-beijing.aliyuncs.com/csghub/all-in-one-CN.yml, the same error.

csghub_server-1  | Database setup...
csghub_server-1  | Migration init
csghub_server-1  | init logger, level: INFO, format: json
csghub_server-1  | Migration migrate
csghub_server-1  | init logger, level: INFO, format: json
csghub_server-1  | {"time":"2024-07-08T08:47:53.771818081Z","level":"INFO","msg":"there are no new migrations to run (database is up to date)"}
csghub_server-1  | Start server...
csghub_server-1  | init logger, level: INFO, format: json
csghub_server-1  | {"time":"2024-07-08T08:47:53.796174633Z","level":"INFO","msg":"FIFOScheduler run started"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.796372517Z","level":"ERROR","msg":"refresh status all failed","error":"Get \"http://localhost:8082/status-all\": dial tcp [::1]:8082: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808899484Z","level":"ERROR","msg":"Error sending request:","!BADKEY":"Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808925276Z","level":"ERROR","msg":"Fail to create mirror server access token","error: ":"Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808928244Z","level":"ERROR","msg":"Failed to find or create token","error: ":"Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | {"time":"2024-07-08T08:47:53.808936128Z","level":"ERROR","msg":"fail to create git mirror server,error:Post \"http://localhost:3001/api/v1/users/root/tokens\": dial tcp [::1]:3001: connect: connection refused"}
csghub_server-1  | Error: failed to init router: error creating user controller:failed to create space component,error:fail to create git mirror server,error:Post "http://localhost:3001/api/v1/users/root/tokens": dial tcp [::1]:3001: connect: connection refused

@pulltheflower 这也是一个 mirror 相关的问题

Jul 17 '24 02:07 Rader

@vagetablechicken please try our latest version

Sep 14 '24 08:09 Rader

@vagetablechicken please try our latest version

Thx, now I can access the web. But csghub_db_init-1, csghub_server_runner-1, server_proxy_nginx-1, the 3 containers are not running, endless restarting.

csghub_server_runner-1 shows

{"time":"2024-09-19T16:11:52.423745644Z","level":"ERROR","msg":"falied to build kubeconfig","error":"invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable"}

I don't have k8s env, so I can skip starting it?

Sep 19 '24 16:09 vagetablechicken