ivory icon indicating copy to clipboard operation
ivory copied to clipboard

Various database query based functions give error 500

Open krisavi opened this issue 7 months ago • 2 comments

Describe the bug Running any query, updating charts or trying to select database, etc under BLOAT tab gives error in log.

To Reproduce Add cluster Add PG password Try to refresh node charts or run query.

Expected behavior Queries to work or more descriptive error why it is failing. Queries are not reaching PG servers, so something is happening before they are being sent.

Desktop (please complete the following information):

  • OS: Doesn't matter, it is browser app
  • Browser: latest Firefox and Chrome
  • Version: v1.3.2 or newer

Additional context

Image

Image

[GIN] 2025/05/14 - 14:11:44 | 500 |     732.837µs |       127.0.0.1 | POST     "/api/query/run"


2025/05/14 14:11:44 [Recovery] 2025/05/14 - 14:11:44 panic recovered:
runtime error: index out of range [2] with length 1
/opt/hostedtoolcache/go/1.22.6/x64/src/runtime/panic.go:114 (0x4383bb)
/home/runner/work/ivory/ivory/service/src/service/postgres.go:305 (0x99fdf7)
/home/runner/work/ivory/ivory/service/src/service/postgres.go:301 (0x99fa29)
/home/runner/work/ivory/ivory/service/src/service/postgres.go:276 (0x99fa0d)
/home/runner/work/ivory/ivory/service/src/service/postgres.go:140 (0x99ec9a)
/home/runner/work/ivory/ivory/service/src/service/postgres.go:93 (0x99e1a4)
/home/runner/work/ivory/ivory/service/src/service/query.go:63 (0x9a06e4)
/home/runner/work/ivory/ivory/service/src/service/query.go:54 (0x9a062b)
/home/runner/work/ivory/ivory/service/src/router/query.go:141 (0x9b0dd2)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x82d1aa)
/home/runner/work/ivory/ivory/service/src/router/auth.go:51 (0x9b6f4e)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x82d1aa)
/home/runner/work/ivory/ivory/service/src/router/secret.go:31 (0x9b7164)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x833179)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 (0x833167)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x833179)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 (0x833167)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x8322a4)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:249 (0x83228b)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x831691)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:633 (0x831100)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:589 (0x830d91)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/h2c/h2c.go:125 (0x828496)
/opt/hostedtoolcache/go/1.22.6/x64/src/net/http/server.go:3142 (0x6e03cd)
/opt/hostedtoolcache/go/1.22.6/x64/src/net/http/server.go:2044 (0x6dc9a7)
/opt/hostedtoolcache/go/1.22.6/x64/src/runtime/asm_amd64.s:1695 (0x46f640)

[GIN] 2025/05/14 - 14:11:48 | 500 |     226.143µs |       127.0.0.1 | POST     "/api/query/activity"


2025/05/14 14:11:48 [Recovery] 2025/05/14 - 14:11:48 panic recovered:
runtime error: index out of range [2] with length 1
/opt/hostedtoolcache/go/1.22.6/x64/src/runtime/panic.go:114 (0x4383bb)
/home/runner/work/ivory/ivory/service/src/service/postgres.go:305 (0x99fdf7)
/home/runner/work/ivory/ivory/service/src/service/postgres.go:301 (0x9a0878)
/home/runner/work/ivory/ivory/service/src/service/query.go:67 (0x9a0862)
/home/runner/work/ivory/ivory/service/src/router/query.go:192 (0x9b1c1b)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x82d1aa)
/home/runner/work/ivory/ivory/service/src/router/auth.go:51 (0x9b6f4e)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x82d1aa)
/home/runner/work/ivory/ivory/service/src/router/secret.go:31 (0x9b7164)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x833179)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 (0x833167)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x833179)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/recovery.go:102 (0x833167)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x8322a4)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/logger.go:249 (0x83228b)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/context.go:185 (0x831691)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:633 (0x831100)
/home/runner/go/pkg/mod/github.com/gin-gonic/[email protected]/gin.go:589 (0x830d91)
/home/runner/go/pkg/mod/golang.org/x/[email protected]/http2/h2c/h2c.go:125 (0x828496)
/opt/hostedtoolcache/go/1.22.6/x64/src/net/http/server.go:3142 (0x6e03cd)
/opt/hostedtoolcache/go/1.22.6/x64/src/net/http/server.go:2044 (0x6dc9a7)
/opt/hostedtoolcache/go/1.22.6/x64/src/runtime/asm_amd64.s:1695 (0x46f640)

krisavi avatar May 14 '25 14:05 krisavi

Hello, thanks for reporting a bug. I think that there is a defenetly some problem with request handling (this is why you get 500), I wonder if it is cruatial for you? and what are your cluster settings setup on Ivory? Bercause I don't have such problem, so probably it is some corner case with some settings setup, I wonder what it cloud be.

anselvo avatar May 15 '25 15:05 anselvo

I am using docker run -p 80:80 --restart always --name ivory -d --mount type=bind,source=/root/ivory/data,target=/opt/data aelsergeev/ivory:latest to start up the docker container. Docker container is running on AlmaLinux, but it should not matter. None of the databases even get any connection, it just fails on application side before sending out the request. At first I thought it is problem with hba or firewall side, but everything worked on older version.

I added bunch of clusters, various PG versions. Patroni itself is using etcd3 for consensus but it should not matter even. The Ivory -> patroni connection works, switchovers work, etc. Ivory -> Database does not for versions that were after 1.3.1. I would suspect something happens at query builder level. I could try to run it from local machine with debugger attached to see what is causing the error.


Seems that the main problem is lack of Authorization token. When you opt out on using basic authentication then you will not have Authorization token. Without it none of the queries work. Reinitializing cluster with it made it work.

krisavi avatar May 21 '25 11:05 krisavi

ok I see it is strange because there was no breaking changes between 1.3.1 and 1.3.6. One thing that I can recommend it is to clear cache via settings it will clear you browser cache and if there was some changes, it will start working, because Ivory save all settings in localStorage there is possability. If event this doesn't help. you can Erase all Ivory data, but you will need to setup it from scratch. Please, try this options and let me know if it helped.

Image

anselvo avatar May 23 '25 12:05 anselvo

I initially installed without basic authentication. So I had no cache to begin with. After it was not working then I started to downgrade and see in which version it did work. Found out 1.3.1 was the latest one where it was still operational.

When running in debug mode then errors seemed to pop up at Authentication token. Did erase all data and created new config, but this time enabled basic auth during initialization and everything worked.

But tried again without enabling basic auth, tried clearing cache, opening in different browsers, private mode, etc. Queries are giving error 500.

krisavi avatar May 28 '25 09:05 krisavi

Ok thanks for the info. I will try to recheck this.

anselvo avatar May 30 '25 09:05 anselvo

Hello, I have reproduced your issue will try to fix it as soon as possible. it would be next release, I will inform here when it is created. Meanwhile I would recommend to use Ivory with auth set :) sorry for inconvinence....

Really appritiate for you contribution ❤

anselvo avatar Jun 07 '25 17:06 anselvo

Was fixed here https://github.com/veegres/ivory/pull/634, will be available in the next realease,

anselvo avatar Jun 23 '25 16:06 anselvo

fix deployed 1.3.7

anselvo avatar Jun 23 '25 17:06 anselvo