server icon indicating copy to clipboard operation
server copied to clipboard

Published 20 hours ago • verified publisher icon nextcloud

[Bug]: Update 22.2.3 --> 22.2.5 or 23.0.2: Database error

Open edes opened this issue 3 years ago • 15 comments

⚠️ This issue respects the following points: ⚠️

  • [ ] This is a bug, not a question or a configuration/webserver/proxy issue.
  • [X] This issue is not already reported on Github (I've searched it).
  • [X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
  • [X] I agree to follow Nextcloud's Code of Conduct.

Bug description

When trying to upgrade from 22.2.3 to 22.2.5 or 23.0.2, occ upgrade (or the web installer) exit with:

Exception: Database error when running migration latest for app core

Steps to reproduce

Login on web UI as admin and update NC via web updater or Downlad and unpack .zip package into install directory, restore original config.php and run occ upgrade

Expected behavior

Database scheme updates w/o errors.

Installation method

Web installer on a VPS or web space

Operating system

RHEL/CentOS

PHP engine version

PHP 7.4

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Updated from a minor version (ex. 22.2.3 to 22.2.4)

Are you using the Nextcloud Server Encryption module?

No response

What user-backends are you using?

  • [X] Default user-backend (database)
  • [ ] LDAP/ Active Directory
  • [ ] SSO - SAML
  • [ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "xx.cc",
            "yy.cc"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "22.2.3.0",
        "overwrite.cli.url": "https:\/\/testcloud.edinger.consulting",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "3306",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "default_language": "de",
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "mysql.utf8mb4": true,
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 12345
        },
        "mail_sendmailmode": "smtp",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "simpleSignUpLink.shown": false,
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "tls",
        "theme": "",
        "loglevel": 2,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "updater.release.channel": "stable",
        "updater.secret": "***REMOVED SENSITIVE VALUE***"
    }
}

List of activated Apps

Enabled:
  - accessibility: 1.8.0
  - activity: 2.15.0
  - admin_audit: 1.12.0
  - cloud_federation_api: 1.5.0
  - comments: 1.12.0
  - contactsinteraction: 1.3.0
  - dav: 1.19.0
  - federatedfilesharing: 1.12.0
  - federation: 1.12.0
  - files: 1.17.0
  - files_external: 1.13.0
  - files_pdfviewer: 2.3.1
  - files_rightclick: 1.1.0
  - files_sharing: 1.14.0
  - files_trashbin: 1.12.0
  - files_versions: 1.15.0
  - files_videoplayer: 1.11.0
  - firstrunwizard: 2.11.0
  - logreader: 2.7.0
  - lookup_server_connector: 1.10.0
  - nextcloud_announcements: 1.11.0
  - notifications: 2.10.1
  - oauth2: 1.10.0
  - password_policy: 1.12.0
  - privacy: 1.6.0
  - provisioning_api: 1.12.0
  - recommendations: 1.1.0
  - serverinfo: 1.12.0
  - settings: 1.4.0
  - systemtags: 1.12.0
  - text: 3.3.0
  - theming: 1.13.0
  - twofactor_backupcodes: 1.11.0
  - updatenotification: 1.12.0
  - user_ldap: 1.12.1
  - user_status: 1.2.0
  - viewer: 1.6.0
  - workflowengine: 2.4.0
Disabled:
  - circles: 0.18.4
  - dashboard: 7.1.0
  - encryption
  - photos: 1.3.0
  - sharebymail: 1.11.0
  - support: 1.4.0
  - survey_client: 1.9.0
  - weather_status: 1.1.0

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

edes avatar Feb 20 '22 21:02 edes

This issue is not already reported on Github (I've searched it).

Do you really searched it? https://github.com/nextcloud/server/search?q=Database+error+when+running+migration+latest+for+app+core&type=issues

solracsf avatar Feb 21 '22 06:02 solracsf

Yes I did this general seach, but there seem to be lotsa possible reasons for errors during database scheme updates. So I looked into nextcloud.log - there I found the error SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

The complete error stack (when trying to upgrade to NC 23.0.2) is:

{"reqId":"YhKx9Pty1NaEiKHKd9u1jgAAAAw","level":3,"time":"2022-02-20T21:26:13+00:00","remoteAddr":"81.217.3.35","user":"--","app":"no app in context","method":"GET","url":"/core/ajax/update.php?requesttoken=znX8xu%2BbIr0Fj%2F%2F5ih%2FhJtfTe0oePiOWfHUtM84sDGg%3D%3AngTM8YPdFPZC1o7IwXKVRZ%2BwLXNLaVXQSjBsBv5dQCo%3D","message":"Database error when running migration latest for app core","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:97.0) Gecko/20100101 Firefox/97.0","version":"22.2.3.0","exception":{"Exception":"Exception","Message":"Database error when running migration latest for app core","Code":0,"Trace":[{"file":"/[myInstallRoot]/lib/private/Updater.php","line":315,"function":"migrate","class":"OC\DB\MigrationService","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":254,"function":"doCoreUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":133,"function":"doUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/core/ajax/update.php","line":194,"function":"upgrade","class":"OC\Updater","type":"->"}],"File":"/[myInstallRoot]/lib/private/DB/MigrationService.php","Line":429,"Previous":{"Exception":"Doctrine\DBAL\Exception\ConnectionLost","Message":"An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away","Code":2006,"Trace":[{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1780,"function":"convert","class":"Doctrine\DBAL\Driver\API\MySQL\ExceptionConverter","type":"->"},{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1719,"function":"handleDriverException","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1067,"function":"convertExceptionDuringQuery","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Connection.php","line":237,"function":"executeQuery","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1809,"function":"executeQuery","class":"OC\DB\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Migrator.php","line":175,"function":"query","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Migrator.php","line":76,"function":"applySchema","class":"OC\DB\Migrator","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Connection.php","line":546,"function":"migrate","class":"OC\DB\Migrator","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/MigrationService.php","line":540,"function":"migrateToSchema","class":"OC\DB\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/MigrationService.php","line":425,"function":"executeStep","class":"OC\DB\MigrationService","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":315,"function":"migrate","class":"OC\DB\MigrationService","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":254,"function":"doCoreUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":133,"function":"doUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/core/ajax/update.php","line":194,"function":"upgrade","class":"OC\Updater","type":"->"}],"File":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php","Line":101,"Previous":{"Exception":"Doctrine\DBAL\Driver\PDO\Exception","Message":"SQLSTATE[HY000]: General error: 2006 MySQL server has gone away","Code":2006,"Trace":[{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Driver/PDO/Connection.php","line":87,"function":"new","class":"Doctrine\DBAL\Driver\PDO\Exception","type":"::"},{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1062,"function":"query","class":"Doctrine\DBAL\Driver\PDO\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Connection.php","line":237,"function":"executeQuery","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1809,"function":"executeQuery","class":"OC\DB\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Migrator.php","line":175,"function":"query","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Migrator.php","line":76,"function":"applySchema","class":"OC\DB\Migrator","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Connection.php","line":546,"function":"migrate","class":"OC\DB\Migrator","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/MigrationService.php","line":540,"function":"migrateToSchema","class":"OC\DB\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/MigrationService.php","line":425,"function":"executeStep","class":"OC\DB\MigrationService","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":315,"function":"migrate","class":"OC\DB\MigrationService","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":254,"function":"doCoreUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":133,"function":"doUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/core/ajax/update.php","line":194,"function":"upgrade","class":"OC\Updater","type":"->"}],"File":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php","Line":26,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[HY000]: General error: 2006 MySQL server has gone away","Code":"HY000","Trace":[{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Driver/PDO/Connection.php","line":82,"function":"query","class":"PDO","type":"->"},{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1062,"function":"query","class":"Doctrine\DBAL\Driver\PDO\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Connection.php","line":237,"function":"executeQuery","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Connection.php","line":1809,"function":"executeQuery","class":"OC\DB\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Migrator.php","line":175,"function":"query","class":"Doctrine\DBAL\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Migrator.php","line":76,"function":"applySchema","class":"OC\DB\Migrator","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/Connection.php","line":546,"function":"migrate","class":"OC\DB\Migrator","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/MigrationService.php","line":540,"function":"migrateToSchema","class":"OC\DB\Connection","type":"->"},{"file":"/[myInstallRoot]/lib/private/DB/MigrationService.php","line":425,"function":"executeStep","class":"OC\DB\MigrationService","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":315,"function":"migrate","class":"OC\DB\MigrationService","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":254,"function":"doCoreUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/lib/private/Updater.php","line":133,"function":"doUpgrade","class":"OC\Updater","type":"->"},{"file":"/[myInstallRoot]/core/ajax/update.php","line":194,"function":"upgrade","class":"OC\Updater","type":"->"}],"File":"/[myInstallRoot]/3rdparty/doctrine/dbal/src/Driver/PDO/Connection.php","Line":82}}},"CustomMessage":"Database error when running migration latest for app core"}}

My MariaDB is running healty w/o any config changes since quite a while and I never ran into (major) issues with NC updates since version 17. Unfortunaltely I'm not into NC development and can't really interpret that error stack - I wonder what the migration routine actually requests from the database when the problem occures?

edes avatar Feb 21 '22 08:02 edes

@acsfer: Is that info sufficient?

edes avatar Feb 24 '22 12:02 edes

So... running journalctl -u mariadb -f during the (failing) update brings up this error journal:

Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: Failing assertion: dict_tf2_is_valid(flags, flags2) Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: We intentionally generate a memory trap. Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/ Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: If you get repeated assertion failures or crashes, even Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: immediately after the mysqld startup, there may be Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: corruption in the InnoDB tablespace. Please refer to Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/ Mar 02 19:28:59 localhost mariadbd[75187]: InnoDB: about forcing recovery. Mar 02 19:28:59 localhost mariadbd[75187]: 220302 19:28:59 [ERROR] mysqld got signal 6 ; Mar 02 19:28:59 localhost mariadbd[75187]: This could be because you hit a bug. It is also possible that this binary Mar 02 19:28:59 localhost mariadbd[75187]: or one of the libraries it was linked against is corrupt, improperly built, Mar 02 19:28:59 localhost mariadbd[75187]: or misconfigured. This error can also be caused by malfunctioning hardware. Mar 02 19:28:59 localhost mariadbd[75187]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs Mar 02 19:28:59 localhost mariadbd[75187]: We will try our best to scrape up some info that will hopefully help Mar 02 19:28:59 localhost mariadbd[75187]: diagnose the problem, but since we have already crashed, Mar 02 19:28:59 localhost mariadbd[75187]: something is definitely wrong and this may fail. Mar 02 19:28:59 localhost mariadbd[75187]: Server version: 10.5.15-MariaDB Mar 02 19:28:59 localhost mariadbd[75187]: key_buffer_size=134217728 Mar 02 19:28:59 localhost mariadbd[75187]: read_buffer_size=131072 Mar 02 19:28:59 localhost mariadbd[75187]: max_used_connections=8 Mar 02 19:28:59 localhost mariadbd[75187]: max_threads=153 Mar 02 19:28:59 localhost mariadbd[75187]: thread_count=8 Mar 02 19:28:59 localhost mariadbd[75187]: It is possible that mysqld could use up to Mar 02 19:28:59 localhost mariadbd[75187]: key_buffer_size + (read_buffer_size + sort_buffer_size)max_threads = 467872 K bytes of memory Mar 02 19:28:59 localhost mariadbd[75187]: Hope that's ok; if not, decrease some variables in the equation. Mar 02 19:28:59 localhost mariadbd[75187]: Thread pointer: 0x7fbcd00009b8 Mar 02 19:28:59 localhost mariadbd[75187]: Attempting backtrace. You can use the following information to find out Mar 02 19:28:59 localhost mariadbd[75187]: where mysqld died. If you see no messages after this, something went Mar 02 19:28:59 localhost mariadbd[75187]: terribly wrong... Mar 02 19:28:59 localhost mariadbd[75187]: stack_bottom = 0x7fbd2d446c90 thread_stack 0x49000 Mar 02 19:28:59 localhost mariadbd[75187]: ??:0(my_print_stacktrace)[0x55a7b173de3e] Mar 02 19:28:59 localhost mariadbd[75187]: ??:0(handle_fatal_signal)[0x55a7b112f9c7] Mar 02 19:29:00 localhost mariadbd[75187]: sigaction.c:0(__restore_rt)[0x7fbd4846a630] Mar 02 19:29:00 localhost mariadbd[75187]: :0(__GI_raise)[0x7fbd478b5387] Mar 02 19:29:00 localhost mariadbd[75187]: :0(__GI_abort)[0x7fbd478b6a78] Mar 02 19:29:00 localhost mariadbd[75187]: /usr/sbin/mariadbd(+0x66b98d)[0x55a7b0dff98d] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(std::pair<std::_Rb_tree_iterator, bool> std::_Rb_tree<unsigned int, unsigned int, std::_Identity, std::less, std::allocator >::_M_insert_unique<unsigned int const&>(unsigned int const&))[0x55a7b1622400] Mar 02 19:29:00 localhost mariadbd[75187]: /usr/sbin/mariadbd(+0x65ef1d)[0x55a7b0df2f1d] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const))[0x55a7b1448b01] Mar 02 19:29:00 localhost mariadbd[75187]: /usr/sbin/mariadbd(+0x6555a2)[0x55a7b0de95a2] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*, unsigned int, st_order*, bool, bool))[0x55a7b0fc4cb3] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(Sql_cmd_alter_table::execute(THD*))[0x55a7b10218de] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(mysql_execute_command(THD*))[0x55a7b0f269c7] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x55a7b0f2b2e5] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x55a7b0f2dbb5] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(do_command(THD*))[0x55a7b0f2f34f] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(do_handle_one_connection(CONNECT*, bool))[0x55a7b101cbc2] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(handle_one_connection)[0x55a7b101ce94] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55a7b13851b7] Mar 02 19:29:00 localhost mariadbd[75187]: pthread_create.c:0(start_thread)[0x7fbd48462ea5] Mar 02 19:29:00 localhost mariadbd[75187]: ??:0(__clone)[0x7fbd4797db0d] Mar 02 19:29:00 localhost mariadbd[75187]: Trying to get some variables. Mar 02 19:29:00 localhost mariadbd[75187]: Some pointers may be invalid and cause the dump to abort. Mar 02 19:29:00 localhost mariadbd[75187]: Query (0x7fbcd0010470): ALTER TABLE oc_jobs ADD argument_hash VARCHAR(32) DEFAULT NULL Mar 02 19:29:00 localhost systemd[1]: mariadb.service: main process exited, code=killed, status=6/ABRT Mar 02 19:29:00 localhost systemd[1]: Unit mariadb.service entered failed state. Mar 02 19:29:00 localhost systemd[1]: mariadb.service failed.

Obviously mariadb failed to execute the query ALTER TABLE oc_jobs ADD argument_hash VARCHAR(32) DEFAULT NULL. The failing dict_tf2_is_valid(flags, flags2) assertion seems to be related to inconsistencies in InnoDB tables.

In my case the solution was to export the (inconsistent?) table's structure and contents to SQL, drop it, and rebuild a similarily looking table from the export - after that, the Nextcloud update went smoothly.

edes avatar Mar 02 '22 21:03 edes

I have the same error - the second update in a row that has had database problems (mariadb first needed to be updated, then didn't work with 10.6, so I had to revert to 10.5)

This is what happens when I try to add the column manually:

Server version: 10.5.15-MariaDB-1:10.5.15+maria~bionic mariadb.org binary distribution

MariaDB [nextcloud]> ALTER TABLE oc_jobs ADD argument_hash VARCHAR(32) DEFAULT NULL;
ERROR 2013 (HY000): Lost connection to MySQL server during query

edonnachie avatar Apr 29 '22 09:04 edonnachie

Hi, i think I got the same problem but with a different nextcloud version. I'm still hijacking this thread but can open a separate issue if desired.

The error occurs when trying an upgrade from Nextcloud 23.0.4.1 to 24.0.0.

$ uname -rsva
Linux 5.13.19-6-pve #1 SMP PVE 5.13.19-15 (Tue, 29 Mar 2022 15:59:50 +0200) x86_64 GNU/Linux

$ mysql --version
mysql  Ver 15.1 Distrib 10.5.15-MariaDB, for debian-linux-gnu (x86_64) using  EditLine wrapper

The command that fails:

$ sudo -u www-data php /var/www/nextcloud/occ upgrade
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
Setting log level to debug
Updating database schema
Exception: Database error when running migration 24000Date20220202150027 for app core
An exception occurred while executing a query: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
Update failed
Maintenance mode is kept active
Resetting log level

From the error log (stacktrace appended below) I can see the Failing assertion: dict_tf2_is_valid(flags, flags2). It seems the failing command was mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*, unsigned int, st_order*, bool, bool) but I don't know about that.

I don't want to execute any sql commands manually if possible. If I understand correctly, to downgrade my mysql installation I need to make an export, apt purge the installation and install a different version. I don't really want to do that, especially because I would not even know how to install a different (minor) version.

 $ apt list -a mariadb-server
Listing... Done
mariadb-server/stable,now 1:10.5.15-0+deb11u1 all [installed]

 $ apt search mariadb-server
Sorting... Done
Full Text Search... Done
mariadb-server/stable,now 1:10.5.15-0+deb11u1 all [installed]
  MariaDB database server (metapackage depending on the latest version)

mariadb-server-10.1/now 10.1.44-0+deb9u1 amd64 [residual-config]
  (none)

mariadb-server-10.3/now 1:10.3.31-0+deb10u1 amd64 [residual-config]
  (none)

mariadb-server-10.5/stable,now 1:10.5.15-0+deb11u1 amd64 [installed,automatic]
  MariaDB database server binaries

mariadb-server-core-10.5/stable,now 1:10.5.15-0+deb11u1 amd64 [installed,automatic]
  MariaDB database core server files

I'm happy to provide more info if required. Any help is greatly appreciated :)

The full stacktrace dumped to the error log (syslog in my case):

mariadbd[1988]: 2022-05-04 11:58:32 0x7f8684806700  InnoDB: Assertion failure in file ./storage/innobase/dict/dict0mem.cc line 147
mariadbd[1988]: InnoDB: Failing assertion: dict_tf2_is_valid(flags, flags2)
mariadbd[1988]: InnoDB: We intentionally generate a memory trap.
mariadbd[1988]: InnoDB: Submit a detailed bug report to https://jira.mariadb.org/
mariadbd[1988]: InnoDB: If you get repeated assertion failures or crashes, even
mariadbd[1988]: InnoDB: immediately after the mysqld startup, there may be
mariadbd[1988]: InnoDB: corruption in the InnoDB tablespace. Please refer to
mariadbd[1988]: InnoDB: https://mariadb.com/kb/en/library/innodb-recovery-modes/
mariadbd[1988]: InnoDB: about forcing recovery.
mariadbd[1988]: 220504 11:58:32 [ERROR] mysqld got signal 6 ;
mariadbd[1988]: This could be because you hit a bug. It is also possible that this binary
mariadbd[1988]: or one of the libraries it was linked against is corrupt, improperly built,
mariadbd[1988]: or misconfigured. This error can also be caused by malfunctioning hardware.
mariadbd[1988]: To report this bug, see https://mariadb.com/kb/en/reporting-bugs
mariadbd[1988]: We will try our best to scrape up some info that will hopefully help
mariadbd[1988]: diagnose the problem, but since we have already crashed,
mariadbd[1988]: something is definitely wrong and this may fail.
mariadbd[1988]: Server version: 10.5.15-MariaDB-0+deb11u1-log
mariadbd[1988]: key_buffer_size=134217728
mariadbd[1988]: read_buffer_size=131072
mariadbd[1988]: max_used_connections=1
mariadbd[1988]: max_threads=153
mariadbd[1988]: thread_count=1
mariadbd[1988]: It is possible that mysqld could use up to
mariadbd[1988]: key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467872 K  bytes of memory
mariadbd[1988]: Hope that's ok; if not, decrease some variables in the equation.
mariadbd[1988]: Thread pointer: 0x7f85fc000c58
mariadbd[1988]: Attempting backtrace. You can use the following information to find out
mariadbd[1988]: where mysqld died. If you see no messages after this, something went
mariadbd[1988]: terribly wrong...
mariadbd[1988]: stack_bottom = 0x7f8684805d78 thread_stack 0x49000
mariadbd[1988]: ??:0(my_print_stacktrace)[0x55aeff3e654e]
mariadbd[1988]: ??:0(handle_fatal_signal)[0x55aefeee5f65]
mariadbd[1988]: ??:0(__restore_rt)[0x7f868c449140]
mariadbd[1988]: ??:0(gsignal)[0x7f868bf92ce1]
mariadbd[1988]: ??:0(abort)[0x7f868bf7c537]
mariadbd[1988]: ??:0(Wsrep_server_service::log_dummy_write_set(wsrep::client_state&, wsrep::ws_meta const&))[0x55aefebc7b4c]
mariadbd[1988]: ??:0(Wsrep_server_service::log_dummy_write_set(wsrep::client_state&, wsrep::ws_meta const&))[0x55aefebd89e0]
mariadbd[1988]: ??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x55aeff1e0927]
mariadbd[1988]: ??:0(wsrep_notify_status(wsrep::server_state::state, wsrep::view const*))[0x55aeff1e3c07]
mariadbd[1988]: ??:0(mysql_alter_table(THD*, st_mysql_const_lex_string const*, st_mysql_const_lex_string const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*, unsigned int, st_order*, bool, bool))[0x55aefed91737]
mariadbd[1988]: ??:0(Sql_cmd_alter_table::execute(THD*))[0x55aefedebc3c]
mariadbd[1988]: ??:0(mysql_execute_command(THD*))[0x55aefeced356]
mariadbd[1988]: ??:0(mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool))[0x55aefecf15db]
mariadbd[1988]: ??:0(dispatch_command(enum_server_command, THD*, char*, unsigned int, bool, bool))[0x55aefecf3a5d]
mariadbd[1988]: ??:0(do_command(THD*))[0x55aefecf52de]
mariadbd[1988]: ??:0(do_handle_one_connection(CONNECT*, bool))[0x55aefede6fb2]
mariadbd[1988]: ??:0(handle_one_connection)[0x55aefede722d]
mariadbd[1988]: ??:0(MyCTX_nopad::finish(unsigned char*, unsigned int*))[0x55aeff12311b]
mariadbd[1988]: ??:0(start_thread)[0x7f868c43dea7]
mariadbd[1988]: ??:0(clone)[0x7f868c054def]
mariadbd[1988]: Trying to get some variables.
mariadbd[1988]: Some pointers may be invalid and cause the dump to abort.
mariadbd[1988]: Query (0x7f85fc012b60): ALTER TABLE oc_mounts ADD mount_provider_class VARCHAR(128) DEFAULT NULL
mariadbd[1988]: Connection ID (thread ID): 32
mariadbd[1988]: Status: NOT_KILLED
mariadbd[1988]: Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=on,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=on,condition_pushdown_for_derived=on,split_materialized=on,condition_pushdown_for_subquery=on,rowid_filter=on,condition_pushdown_from_having=on,not_null_range_scan=off
mariadbd[1988]: The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains
mariadbd[1988]: information that should help you find out what is causing the crash.
mariadbd[1988]: Writing a core file...
mariadbd[1988]: Working directory at /var/lib/mysql
mariadbd[1988]: Resource Limits:
mariadbd[1988]: Limit                     Soft Limit           Hard Limit           Units
mariadbd[1988]: Max cpu time              unlimited            unlimited            seconds
mariadbd[1988]: Max file size             unlimited            unlimited            bytes
mariadbd[1988]: Max data size             unlimited            unlimited            bytes
mariadbd[1988]: Max stack size            8388608              unlimited            bytes
mariadbd[1988]: Max core file size        0                    unlimited            bytes
mariadbd[1988]: Max resident set          unlimited            unlimited            bytes
mariadbd[1988]: Max processes             120149               120149               processes
mariadbd[1988]: Max open files            32768                32768                files
mariadbd[1988]: Max locked memory         65536                65536                bytes
mariadbd[1988]: Max address space         unlimited            unlimited            bytes
mariadbd[1988]: Core pattern: core
systemd[1]: mariadb.service: Main process exited, code=killed, status=6/ABRT
systemd[1]: mariadb.service: Failed with result 'signal'.
systemd[1]: mariadb.service: Consumed 1.532s CPU time.
systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 1.
systemd[1]: Stopped MariaDB 10.5.15 database server.
systemd[1]: mariadb.service: Consumed 1.532s CPU time.
systemd[1]: Starting MariaDB 10.5.15 database server...
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] /usr/sbin/mariadbd (mysqld 10.5.15-MariaDB-0+deb11u1-log) starting as process 2316 ...
mariadbd[2316]: 2022-05-04 11:58:38 0 [Warning] You need to use --log-bin to make --binlog-format work.
mariadbd[2316]: 2022-05-04 11:58:38 0 [Warning] The parameter innodb_file_format is deprecated and has no effect. It may be removed in future releases. See https://mariadb.com/kb/en/library/xtradbinnodb-file-format/
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Uses event mutexes
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Number of pools: 1
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Using Linux native AIO
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Initializing buffer pool, total size = 1073741824, chunk size = 134217728
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Completed initialization of buffer pool
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=605710966108,605710966108
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: 128 rollback segments are active.
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Creating shared tablespace for temporary tables
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: 10.5.15 started; log sequence number 605710966120; transaction id 238190121
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] Plugin 'FEEDBACK' is disabled.
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] Server socket created on IP: '127.0.0.1'.
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] Reading of all Master_info entries succeeded
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] Added new Master_info '' to hash table
mariadbd[2316]: 2022-05-04 11:58:38 0 [Note] /usr/sbin/mariadbd: ready for connections.
mariadbd[2316]: Version: '10.5.15-MariaDB-0+deb11u1-log'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Debian 11
systemd[1]: Started MariaDB 10.5.15 database server.
/etc/mysql/debian-start[2334]: Upgrading MySQL tables if necessary.
/etc/mysql/debian-start[2337]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
/etc/mysql/debian-start[2337]: Looking for 'mysql' as: /usr/bin/mysql
/etc/mysql/debian-start[2337]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
/etc/mysql/debian-start[2337]: This installation of MariaDB is already upgraded to 10.5.12-MariaDB.
/etc/mysql/debian-start[2337]: There is no need to run mysql_upgrade again for 10.5.15-MariaDB.
/etc/mysql/debian-start[2337]: You can use --force if you still want to run mysql_upgrade
/etc/mysql/debian-start[2345]: Checking for insecure root accounts.
/etc/mysql/debian-start[2349]: Triggering myisam-recover for all MyISAM tables and aria-recover for all Aria tables
mariadbd[2316]: 2022-05-04 11:58:39 0 [Note] InnoDB: Buffer pool(s) load completed at 220504 11:58:39

kantholtz avatar May 04 '22 10:05 kantholtz

To correct my former post - the failing query was ALTER TABLE oc_mounts ADD mount_provider_class VARCHAR(128) DEFAULT NULL

I got it working without finding out the root cause (just like @edes) by simply exporting, deleting and importing the database:

$ mysqldump --single-transaction -u [USER] [DATABASE] > /root/2022.05.22-nextcloud.sql
$ mysql -p [DATABASE] -u [USER]
MariaDB [nextcloud]> drop database [DATABASE];
MariaDB [(none)]> create database [DATABASE];
MariaDB [(none)]> ^DBye
$ mysql -u [USER] [DATABASE] < 2022.05.22-nextcloud.sql
$ sudo -u www-data php /var/www/nextcloud/occ upgrade
$ sudo -u www-data php /var/www/nextcloud/occ db:add-missing-indices
$ sudo -u www-data php /var/www/nextcloud/occ maintenance:mode --off

Et voila.

kantholtz avatar May 05 '22 12:05 kantholtz

Same problem while updating from NC 23.0.4 -> 24.0.0 The solution from kantholz did the trick.

TheRaven500 avatar May 06 '22 19:05 TheRaven500

Same problem while updating from NC 23.0.5 -> 24.0.1. The solution from kantholz also helped me. Although I was using web-based update and I skipped last 3 commands (but still had to perform db:add-missing-indices manually after upgrade)

GritsanY avatar May 26 '22 07:05 GritsanY

Same here, Updating vom latest 23.x to 24.x Dropping and re-creating database did it. Thanks.

maxir3 avatar Jun 22 '22 08:06 maxir3

I just encountered the same problem. I did this: I opened the table mount_provider_class in phpmyadmin and then went to operations and did "Analyze table", "Check table" and "Optimize table". After that, the update worked.

vaspervnp avatar Jun 27 '22 11:06 vaspervnp

I can confirm that the method used by vaspervnp works (thank you!).

I used mysqlcheck to clean the database as follows:

mysqlcheck -u nextcloud -p --databases nextcloud --optimize
mysqlcheck -u nextcloud -p --databases nextcloud

The optimize stage was necessary before the check, otherwise the check stage ended in failure in a similar way to the nextcloud update. Having performed these two commands, upgrade was successful.

edonnachie avatar Jun 30 '22 09:06 edonnachie

I had the same issue upgrading from 23.x to 24.x. Solution from @kantholtz worked for me.

snowtr avatar Jul 04 '22 18:07 snowtr

@edonnachie answer worked for me (simple and less risky)

phpbg avatar Jul 31 '22 16:07 phpbg

I can confirm that the method used by vaspervnp works (thank you!).

I used mysqlcheck to clean the database as follows:

mysqlcheck -u nextcloud -p --databases nextcloud --optimize
mysqlcheck -u nextcloud -p --databases nextcloud

The optimize stage was necessary before the check, otherwise the check stage ended in failure in a similar way to the nextcloud update. Having performed these two commands, upgrade was successful.

you can even check all databases: mysqlcheck -u nextcloud -p -A --optimize

This did the trick for me !

cramaboule avatar Aug 05 '22 07:08 cramaboule

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

nextcloud-command avatar Sep 06 '22 00:09 nextcloud-command