calendar
calendar copied to clipboard
Published
20 hours ago •
nextcloud
nextcloud
export then import calender partially fails (error: 5 database is locked)
Steps to reproduce
- I was using an upgraded nextcloud 27.1.5 instance. I have a calendar full of annual recurring events (brithdays etc) called yearly
- export the calendar "yearly" to a file
- create a new calendar called test1
- try to import the just created exported calendar file into test1
- You will get pop showing a different number of success out total each time you try!
- Looking in the logfile you will see a load errors
notes: same when trying to import the exported file from current instance into a fresh install of 28.0.1 or fresh install of 27.1.5 as well as using the current version that was updated from 27.1.4.
Expected behavior
you should get popup showing alll the entries where imported, and no errors (I assume) in the logfile
Actual behaviour
You get a pop up the shows only a subset of the entries were inported and you get errors in the log, which changes with each attempt: Mainly: OC\DB\Exceptions\DbalException: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 5 database is locked also: Sabre\VObject\Recur\MaxInstancesExceededException: Recurring events are only allowed to generate 3500 Also: Sabre\DAV\Exception\UnsupportedMediaType: Validation error in iCalendar: More than one master object was found for the object with UID 3c1b8ebf-1bf0-4bbe-abdc-8b560e988c4e
Calendar app version
4.6.1
CalDAV-clients used
nextcloud 27.1.5 and 28.0.1
Browser
Firefox 121.0 (64bit)
Client operating system
Windows 11 (patched)
Server operating system
ARM Linux (rpi4 64bit)
Web server
Apache
Database engine version
SQlite
PHP engine version
PHP 8.2
Nextcloud version
27.1.5 and 28.0.1
Updated from an older installed version or fresh install
Fresh install
List of activated apps
root@421b374e64f6:/var/www/nextcloud# sudo -u www-data php occ app:list
Cannot load Zend OPcache - it was already loaded
Enabled:
- activity: 2.19.0
- bookmarks: 13.1.3
- bruteforcesettings: 2.7.0
- calendar: 4.6.1
- circles: 27.0.1
- cloud_federation_api: 1.10.0
- comments: 1.17.0
- contacts: 5.5.0
- contactsinteraction: 1.8.0
- dashboard: 7.7.0
- dav: 1.27.0
- federatedfilesharing: 1.17.0
- federation: 1.17.0
- files: 1.22.0
- files_pdfviewer: 2.8.0
- files_reminders: 1.0.0
- files_rightclick: 1.6.0
- files_sharing: 1.19.0
- files_trashbin: 1.17.0
- files_versions: 1.20.0
- firstrunwizard: 2.16.0
- logreader: 2.12.0
- lookup_server_connector: 1.15.0
- nextcloud_announcements: 1.16.0
- notes: 4.9.1
- notifications: 2.15.0
- oauth2: 1.15.1
- password_policy: 1.17.0
- photos: 2.3.0
- privacy: 1.11.0
- provisioning_api: 1.17.0
- recommendations: 1.6.0
- related_resources: 1.2.0
- serverinfo: 1.17.0
- settings: 1.9.0
- sharebymail: 1.17.0
- support: 1.10.0
- survey_client: 1.15.0
- systemtags: 1.17.0
- tasks: 0.15.0
- text: 3.8.0
- theming: 2.2.0
- twofactor_backupcodes: 1.16.0
- updatenotification: 1.17.0
- user_status: 1.7.0
- viewer: 2.1.0
- weather_status: 1.7.0
- workflowengine: 2.9.0
Disabled:
- admin_audit: 1.17.0
- encryption: 2.15.0
- files_external: 1.19.0
- suspicious_login: 5.0.0
- twofactor_totp: 9.0.0
- user_ldap: 1.17.0
Nextcloud configuration
root@421b374e64f6:/var/www/nextcloud# sudo -u www-data php occ config:list system
Cannot load Zend OPcache - it was already loaded
{
"system": {
"overwrite.cli.url": "https:\/\/xxxxxxx.yyyyyyyy.zzzzzz:2814",
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"xxxxxxx.yyyyyyyy.zzzzzz:2814"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "sqlite3",
"version": "27.1.5.1",
"installed": true
}
}
Web server error log
[Sat Dec 23 23:34:07.042328 2023] [mpm_prefork:notice] [pid 31] AH00163: Apache/2.4.57 (Debian) OpenSSL/3.0.11 configured -- resuming normal operations
[Sat Dec 23 23:34:07.107336 2023] [core:notice] [pid 31] AH00094: Command line: '/usr/sbin/apache2'
Cannot load Zend OPcache - it was already loaded
[Sat Dec 23 23:41:24.625885 2023] [mpm_prefork:notice] [pid 31] AH00169: caught SIGTERM, shutting down
Cannot load Zend OPcache - it was already loaded
[Sat Dec 23 23:43:22.160472 2023] [mpm_prefork:notice] [pid 29] AH00163: Apache/2.4.57 (Debian) OpenSSL/3.0.11 configured -- resuming normal operations
[Sat Dec 23 23:43:22.188066 2023] [core:notice] [pid 29] AH00094: Command line: '/usr/sbin/apache2'
Cannot load Zend OPcache - it was already loaded
Cannot load Zend OPcache - it was already loaded
Log file
{"reqId":"R1UgtO11VYGdDXxNGm03","level":3,"time":"2023-12-27T16:57:17+00:00","remoteAddr":"192.168.20.240","user":"russel2Za","app":"webdav","method":"PUT","url":"/remote .php/dav/calendars/russel2Za/yearly/B3B79815-8189-44E1-AB37-37724915FF43.ics","message":"An exception occurred while executing a query: SQLSTATE[HY000]: General error: 5 database is locked","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0","version":"27.1.5.1","exception":{"Exception":"OC\\DB\\ Exceptions\\DbalException","Message":"An exception occurred while executing a query: SQLSTATE[HY000]: General error: 5 database is locked","Code":5,"Trace":[{"file":"/var /www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":328,"function":"wrap","class":"OC\\DB\\Exceptions\\DbalException","type":"::"},{"file":"/var/www/nextc loud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1280,"function":"executeStatement","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextclou d/lib/public/AppFramework/Db/TTransactional.php","line":63,"function":"OCA\\DAV\\CalDAV\\{closure}","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->","args":["*** sen sitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1231,"function":"atomic","class":"OCA\\DAV\\CalDAV\\CalDavBack end","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Calendar.php","line":199,"function":"createCalendarObject","class":"OCA\\DAV\\CalDAV\\CalDavBa ckend","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1098,"function":"createFile ","class":"Sabre\\CalDAV\\Calendar","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php"," line":504,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/ev ent/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Serv er.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"i nvokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DA V\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":365,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/next cloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":172,"args":["/var /www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/lib/private/DB/Exceptions/DbalException.php","Line":71,"Previous":{ "Exception":"Doctrine\\DBAL\\Exception\\LockWaitTimeoutException","Message":"An exception occurred while executing a query: SQLSTATE[HY000]: General error: 5 database is locked","Code":5,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1814,"function":"convert","class":"Doctrine\\DBAL\\Driver\\API\\SQ Lite\\ExceptionConverter","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1749,"function":"handleDriverException","class":"Doct rine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1163,"function":"convertExceptionDuringQuery","class":" Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":295,"function":"executeStatement","class":"Doctrine\\DBAL\\Conn ection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":354,"function":"executeStatement","class":"OC\\DB\\Connection", "type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":280,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type ":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":326,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->" },{"file":"/var/www/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1280,"function":"executeStatement","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{ "file":"/var/www/nextcloud/lib/public/AppFramework/Db/TTransactional.php","line":63,"function":"OCA\\DAV\\CalDAV\\{closure}","class":"OCA\\DAV\\CalDAV\\CalDavBackend","ty pe":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1231,"function":"atomic","class":"OCA \\DAV\\CalDAV\\CalDavBackend","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Calendar.php","line":199,"function":"createCalendarObject","class":"O CA\\DAV\\CalDAV\\CalDavBackend","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":10 98,"function":"createFile","class":"Sabre\\CalDAV\\Calendar","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/ lib/DAV/CorePlugin.php","line":504,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/ne xtcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdpar ty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php" ,"line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":" start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":365,"function":"exec","class":"Sabre\\DAV\\Server","type":"-> "},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php" ,"line":172,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/API/SQLi te/ExceptionConverter.php","Line":35,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDO\\Exception","Message":"SQLSTATE[HY000]: General error: 5 database is locked","Co de":5,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","line":103,"function":"new","class":"Doctrine\\DBAL\\Driver\\PDO\\Exceptio n","type":"::"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1153,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDO\\Statement"," type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":295,"function":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file": "/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":354,"function":"executeStatement","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/ww w/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":280,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nex tcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":326,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud /apps/dav/lib/CalDAV/CalDavBackend.php","line":1280,"function":"executeStatement","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/li b/public/AppFramework/Db/TTransactional.php","line":63,"function":"OCA\\DAV\\CalDAV\\{closure}","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->","args":["*** sensiti ve parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1231,"function":"atomic","class":"OCA\\DAV\\CalDAV\\CalDavBackend" ,"type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Calendar.php","line":199,"function":"createCalendarObject","class":"OCA\\DAV\\CalDAV\\CalDavBacken d","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1098,"function":"createFile","c lass":"Sabre\\CalDAV\\Calendar","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line ":504,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/event/ lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.p hp","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invok eMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\S erver","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":365,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextclou d/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":172,"args":["/var/www /nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php","Line":30,"Previou s":{"Exception":"PDOException","Message":"SQLSTATE[HY000]: General error: 5 database is locked","Code":"HY000","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal /src/Driver/PDO/Statement.php","line":101,"function":"execute","class":"PDOStatement","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php", "line":1153,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDO\\Statement","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":295,"func tion":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->"},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":354,"func tion":"executeStatement","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":280,"function":"execut e","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":326,"function":"execute","c lass":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/CalDAV/CalDavBackend.php","line":1280,"function":"executeStatement","clas s":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->"},{"file":"/var/www/nextcloud/lib/public/AppFramework/Db/TTransactional.php","line":63,"function":"OCA\\DAV\\CalDAV\\{c losure}","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/dav/lib/CalDAV/CalDavBac kend.php","line":1231,"function":"atomic","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/CalDAV/Calendar.php"," line":199,"function":"createCalendarObject","class":"OCA\\DAV\\CalDAV\\CalDavBackend","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/next cloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1098,"function":"createFile","class":"Sabre\\CalDAV\\Calendar","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":504,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->","args":["*** sen sitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CoreP lugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/va r/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty /sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":365,"fu nction":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server ","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":172,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/ww w/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","Line":101}}},"message":"An exception occurred while executing a query: SQLSTATE[HY000]: General error: 5 database is locked","exception":{},"CustomMessage":"An exception occurred while executing a query: SQLSTATE[HY000]: General error: 5 database is locked"}}
Browser log
No response
Additional info
the log file is massive, so tried to get one of the database is locked examples, if you need more we can arrange a private transfer if you would like. but as this is easily reproducible and to me looks like a race condition as the server is a slow rpi runing sqlite, maybe the exception should be caught and if locked, a delay and retry a few times then hard exception raised? but I am not a php programmer so couldn't really work out quite what was happening, but I have seen such issues in java code before :-)
What do you need from me for this, or is there documentation to say what I need to look at for configuration. Or should this be posted in the server code base?
Can you please reformat your loglines without the superfluous whitespace, then I can work with the stack trace.
As for Sabre\VObject\Recur\MaxInstancesExceededException: Recurring events are only allowed to generate 3500 this exception is thrown when your recurring events have no end date. We only allow mx 3500 recurrance instances, otherwise they would be endless and you'd possibly create millions of lines in your DB for reminders etc. If you have a stacktrace, it would be great to know if that is from the ReminderService as I suspect or if it's in the Sabre code.
Can you search your exported ics file for 3c1b8ebf-1bf0-4bbe-abdc-8b560e988c4e and copypaste the found results? Everyting between BEGIN:VEVENT and END:VEVENT. Or you can send me the whole ics via email if you like - [email protected].
Same thing is happening to me. Seems related to this issue https://github.com/owncloud/core/issues/12429
Adding to /config/php/www2.conf
pm.max_children = 1
pm.max_spare_servers = 1
pm.min_spare_servers = 1
pm.start_servers = 1
"fixes" it. The import happens pretty slow, (the entire app is much slower) but it actually imports everything perfectly and consitently instead of failing on ~40% of them.
