Upgrading from v17.11.2-ee to v18-ee

[Naushikha]

Hi there,

I've been trying to upgrade my instance from v17.11.2-ee to v18-ee but it fails with a database connection issue.

Recipe: gitlab::database_migrations
  * ruby_block[check remote PG version] action nothing (skipped due to action :nothing)
  * rails_migration[gitlab-rails] action run[2025-05-20T03:16:33+00:00] WARN: gitlab-rails does not have a log_group or default logdir mode defined. Setting to 0700.
    * bash_hide_env[migrate gitlab-rails database] action run
      [execute] rake aborted!
                ActiveRecord::DatabaseConnectionError: There is an issue connecting to your database with your username/password, username: gitlab.
                
                Please check your database configuration to ensure the username/password are valid.
                /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:124:in `read_write'
                /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
                /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:179:in `configure_database'
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:156:in `configure_pg_databases'
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:102:in `block (3 levels) in <top (required)>'
                /opt/gitlab/embedded/bin/bundle:25:in `load'
                /opt/gitlab/embedded/bin/bundle:25:in `<main>'
                
                Caused by:
                PG::ConnectionBad: connection to server on socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432" failed: Connection refused
                	Is the server running locally and accepting connections on that socket?
                /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:124:in `read_write'
                /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
                /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:179:in `configure_database'
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:156:in `configure_pg_databases'
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:102:in `block (3 levels) in <top (required)>'
                /opt/gitlab/embedded/bin/bundle:25:in `load'
                /opt/gitlab/embedded/bin/bundle:25:in `<main>'
                Tasks: TOP => gitlab:db:configure
                (See full trace by running task with --trace)

Is this a known issue? I am simply replacing the image that I am using (v17.11.2-ee) with the latest ee (v18) and recreating the container. Is there something else to be done here that I am missing out on?

[Giattuck]

I just updated two instance from 17.11.1 to 18.0.0 without issue...

[zengxs]

Hi there,

I did a quick search online, and it seems this is a known issue with GitLab itself that can sometimes occur during upgrades. This forum post might be helpful:

https://forum.gitlab.com/t/installing-gitlab-ce-fails-due-to-postgresql-connection-issue/91419

Important tip: Please make sure to back up your data before performing any operations.

[Naushikha]

I just updated two instance from 17.11.1 to 18.0.0 without issue...

I suspect that because I did a back-to-back upgrade to 17.11.1 then trying immediately to 18 (after making sure that 17.11.1 works, of course) might have been an issue... 🤔 Maybe they had some hidden background migrations to the latest Postgres 16? I will retry tonight and update this thread.

Hi there,

I did a quick search online, and it seems this is a known issue with GitLab itself that can sometimes occur during upgrades. This forum post might be helpful:

https://forum.gitlab.com/t/installing-gitlab-ce-fails-due-to-postgresql-connection-issue/91419

Important tip: Please make sure to back up your data before performing any operations.

Well, according to this article he deleted everything related to GitLab, essentially starting from a clean slate, well that would always be my last resort I guess, but I am trying to get this upgraded without having to go through the trouble of a fresh install 😁 (have to backup and restore the many user ssh keys, secrets and all the hidden stuff that GitLab forgets to backup in 'gitlab-backup create', such a hassle)

[Naushikha]

👋🏼 Ahoy there, I come bearing happy news. I have solved my issue. For any other poor soul that comes across something similar, make sure that your embedded Postgres version is 16 Mine was 14, even though I had GitLab 17 (I have been upgrading this instance through several versions). Use gitlab-psql --version to check the current version and use gitlab-ctl pg-upgrade. You should perform the db upgrade in your working version (in my case GitLab 17) and ensure that everything works, then perform the GitLab version upgrade. Everything was a smooth sail there after.

P.S. Thanks for the work on this image @zengxs

[zengxs]

@Naushikha Awesome, glad to hear you got it sorted! And thanks a ton for sharing the solution and detailed steps – this will definitely help others running into similar issues.

[kubrickfr]

Thank you so much @Naushikha !