nextcloud
[Bug]: Can't upgrade because of "Segmentation fault" (and server is not running)
⚠️ This issue respects the following points: ⚠️
- [X] This is a bug, not a question or a configuration/webserver/proxy issue.
- [X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- [X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- [X] I agree to follow Nextcloud's Code of Conduct.
Bug description
I upgraded to the latest minor version and then to the latest next major version, but it fails with "Segmentation fault". And the nginx proxy can don't find any fpm result to serve (and shows 502 Bad Gateway)
I have no idea whats wrong. The acpu config is correct. Redis seems to be also reachable.
./occ upgrade -vvv Nextcloud or one of the apps require upgrade - only a limited number of commands are available You may use your browser or the occ upgrade command to do the upgrade 2024-07-11T20:42:49+00:00 Setting log level to debug 2024-07-11T20:42:49+00:00 Repair step: Repair MySQL collation 2024-07-11T20:42:49+00:00 Repair info: All tables already have the correct collation -> nothing to do 2024-07-11T20:42:49+00:00 Repair step: Repair SQLite autoincrement 2024-07-11T20:42:49+00:00 Repair step: Copy data from accounts table when migrating from ownCloud 2024-07-11T20:42:50+00:00 Repair step: Drop account terms table when migrating from ownCloud 2024-07-11T20:42:50+00:00 Updating database schema 2024-07-11T20:42:50+00:00 Updated database 2024-07-11T20:42:50+00:00 Updating <lookup_server_connector> ... Segmentation fault
Steps to reproduce
Expected behavior
No error (Segmentation fault) and served content via fpm.
Installation method
Community Docker image
Nextcloud Server version
27
Operating system
Other
PHP engine version
PHP 8.2
Web server
Nginx
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Upgraded to a MAJOR version (ex. 28 to 29)
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- [X] Default user-backend (database)
- [ ] LDAP/ Active Directory
- [ ] SSO - SAML
- [ ] Other
Configuration report
{
"system": {
"memcache.local": "\\OC\\Memcache\\APCu",
"apps_paths": [
{
"path": "\/var\/www\/html\/apps",
"url": "\/apps",
"writable": false
},
{
"path": "\/var\/www\/html\/custom_apps",
"url": "\/custom_apps",
"writable": true
}
],
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"domain.tld"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "mysql",
"version": "26.0.13.1",
"overwrite.cli.url": "https:\/\/domain.tld",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"maintenance": true,
"mail_smtpmode": "smtp",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_sendmailmode": "smtp",
"mail_smtpport": "587",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"trusted_proxies": "***REMOVED SENSITIVE VALUE***",
"loglevel": 0,
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"password": "***REMOVED SENSITIVE VALUE***",
"port": 6379
}
}
}
List of activated Apps
Enabled:
- cloud_federation_api: 1.9.0
- dav: 1.25.0
- federatedfilesharing: 1.16.0
- files: 1.21.1
- lookup_server_connector: 1.14.0
- oauth2: 1.14.2
- provisioning_api: 1.16.0
- settings: 1.8.0
- theming: 2.1.1
- twofactor_backupcodes: 1.15.0
- viewer: 1.10.0
- workflowengine: 2.8.0
Nextcloud Signing status
Not possible, server is not starting and shows "502 Bad Gateway"
Nextcloud Logs
Doesnt look revelant to error:
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:05+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::setDebugLogLevel: Set log level to debug","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":0,"time":"2024-07-11T20:37:05+00:00","remoteAddr":"","user":"--","app":"core","method":"","url":"--","message":"starting upgrade from 26.0.13.1 to 27.1.11.3","userAgent":"--","version":"26.0.13.1","data":{"app":"core"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:05+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"OC\\Repair\\Events\\RepairStepEvent: Repair step: Repair MySQL collation","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:05+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"OC\\Repair\\Events\\RepairInfoEvent: Repair info: All tables already have the correct collation -> nothing to do","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:05+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"OC\\Repair\\Events\\RepairStepEvent: Repair step: Repair SQLite autoincrement","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:05+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"OC\\Repair\\Events\\RepairStepEvent: Repair step: Copy data from accounts table when migrating from ownCloud","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:06+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"OC\\Repair\\Events\\RepairStepEvent: Repair step: Drop account terms table when migrating from ownCloud","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:06+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::dbUpgradeBefore: Updating database schema","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:06+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::dbUpgrade: Updated database","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"PKSrZtXcZ5c12GAKyncb","level":1,"time":"2024-07-11T20:37:06+00:00","remoteAddr":"","user":"--","app":"updater","method":"","url":"--","message":"\\OC\\Updater::appUpgradeStarted: Updating <lookup_server_connector> ...","userAgent":"--","version":"26.0.13.1","data":{"app":"updater"}}
{"reqId":"3MRtPxQFz6w5KeqO4wGV","level":0,"time":"2024-07-11T20:40:00+00:00","remoteAddr":"","user":"--","app":"cron","method":"","url":"--","message":"Update required, skipping cron","userAgent":"--","version":"26.0.13.1","data":{"app":"cron"}}
{"reqId":"6JcJFhHrVdptY2egjHfZ","level":0,"time":"2024-07-11T20:40:00+00:00","remoteAddr":"","user":"--","app":"cron","method":"","url":"--","message":"Update required, skipping cron","userAgent":"--","version":"26.0.13.1","data":{"app":"cron"}}
Additional info
./occ status Nextcloud or one of the apps require upgrade - only a limited number of commands are available You may use your browser or the occ upgrade command to do the upgrade
- installed: true
- version: 27.1.11.3
- versionstring: 27.1.11
- edition:
- maintenance: true
- needsDbUpgrade: true
- productname: Nextcloud
- extendedSupport: false
"version": "26.0.13.1",
Which image, precisely, are using?
Please share your Docker Compose file and precisely how you upgraded from whatever version you started this latest upgrade attempt with.
"version": "26.0.13.1",Which image, precisely, are using?
Please share your Docker Compose file and precisely how you upgraded from whatever version you started this latest upgrade attempt with.
You can find the files here:
- https://github.com/TechupBusiness/simple-docker-multi-project/blob/master/applications/system-services/main/nextcloud/app.Dockerfile
- https://github.com/TechupBusiness/simple-docker-multi-project/blob/master/applications/system-services/main/nextcloud/docker-compose.yml
Image (via configuration) from 26-fpm-alpine to 27-fpm-alpine.
Upgrade is described here: https://github.com/TechupBusiness/simple-docker-multi-project/blob/master/applications/system-services/main/nextcloud/Readme.md
Would be helpful if there would be a logfile thats shows the real reason for this error.
You're using a highly customized setup/image. This does not happen in a standard install or using a standard image. I can't even begin to decipher what is going on there without digging into whatever abstractions and customization you've done (and that's not realistic through this channel).
The most useful output would been the Docker container output from the very first start-up of your app container when attempting to upgrade. The upgrade is handled (in the unmodified image at least) within the entrypoint. There is no need to run occ upgrade independently (and if you need to, it's a sign something has already gone wrong).
Would be helpful if there would be a logfile thats shows the real reason for this error.
Well, it's a low-level error that is unlikely to be coming from Nextcloud itself. You'll likely have to troubleshoot what is causing it in your local environment. A million things can cause segfaults.
I will say that the custom GID stuff you're doing may not be compatible with assumptions made within the stock image's entrypoint.sh. The ownership of newer versions of Server that get deployed there (via rsync) may not match your customized Dockerfile's assumptions.
If I was in your position I'd probably eliminate all the variables I possibly can by going to a stock image then working from there incrementally until the problem comes up again. And if it comes up in the stock image, then I'd look at what's notable about the host (OS, Docker platform) to try to find a way to make it possible for someone else to reproduce the same behavior.
You're using a highly customized setup/image. This does not happen in a standard install or using a standard image. I can't even begin to decipher what is going on there without digging into whatever abstractions and customization you've done (and that's not realistic through this channel).
Erm you are aware that the only difference to the default image is, that I set explicit the user/group? Whats highly customized here? I dont have custom entrypoints and also nothing else "custom" with this image!? So its pretty standard image.
The most useful output would been the Docker container output from the very first start-up of your app container when attempting to upgrade. The upgrade is handled (in the unmodified image at least) within the entrypoint. There is no need to run
occ upgradeindependently (and if you need to, it's a sign something has already gone wrong).
Yes I know but it didnt work, same error. Thats why I tried it manually.
The log looked like:
app_1 | Configuring Redis as session handler
app_1 | Initializing nextcloud 26.0.13.1 ...
app_1 | Upgrading nextcloud from 25.0.13.2 ...
app_1 | => Searching for scripts (*.sh) to run, located in the folder: /docker-entrypoint-hooks.d/pre-upgrade
app_1 | Nextcloud or one of the apps require upgrade - only a limited number of commands are available
app_1 | You may use your browser or the occ upgrade command to do the upgrade
app_1 | Setting log level to debug
app_1 | Turned on maintenance mode
app_1 | Updating database schema
app_1 | Updated database
app_1 | Updating <lookup_server_connector> ...
app_1 | Segmentation fault
app_1 | Configuring Redis as session handler
app_1 | => Searching for scripts (*.sh) to run, located in the folder: /docker-entrypoint-hooks.d/before-starting
app_1 | [11-Jul-2024 20:28:23] NOTICE: fpm is running, pid 1
app_1 | [11-Jul-2024 20:28:23] NOTICE: ready to handle connections
Would be helpful if there would be a logfile thats shows the real reason for this error.
Well, it's a low-level error that is unlikely to be coming from Nextcloud itself. You'll likely have to troubleshoot what is causing it in your local environment. A million things can cause segfaults.
Yeah but something must trigger this error... it seems quite high-level to me. Other users reports it was related to outdated extensions etc.
I will say that the custom GID stuff you're doing may not be compatible with assumptions made within the stock image's entrypoint.sh. The ownership of newer versions of Server that get deployed there (via rsync) may not match your customized Dockerfile's assumptions.
Do you know whats the proper ownership? Because I didnt change anything else.
If I was in your position I'd probably eliminate all the variables I possibly can by going to a stock image then working from there incrementally until the problem comes up again. And if it comes up in the stock image, then I'd look at what's notable about the host (OS, Docker platform) to try to find a way to make it possible for someone else to reproduce the same behavior.
I'm using the stock image except of setting ownership (which is needed because I mount existing files too). Linux filesystem stuff. I'm considering now a fresh install because I dont use nextcloud for anything else than just accessing files via app or sharing via web (I dont need existing active shares). But this error is maybe not even related to such things, therefore... I guess I need to know about proper file ownership :)
I remember having similar weird issue due testing some of the images. The safest way of re-starting this is doing docker compose down followed by a docker compose up -d I have no idea where that version 25 came from, but it worked. If it does not help, run all containers that you want to keep, and do a docker system prune. This also helps, but keep in mind that by doing so, all stopped containers / anything not in use at the moment(edit: volumes included) of execution is going to be deleted.
Hi @TechupBusiness -
Our image is derived from the PHP Official Images. Perhaps you'll see something helpful in the upstream mentions of similar errors:
https://github.com/search?q=repo%3Adocker-library%2Fphp+%22segmentation+fault%22&type=issues
We're not equipped to debug these sorts of things here. This channel isn't for general support, but only for bug reporting in our own images. We don't have the bandwidth for it.
In addition to the above repository search, I suggest perhaps posting your query over at the Nextcloud help Forum -https://help.nextcloud.com or possibly the Docker or Alpine forums.