[Bug]: Syncing (shared) calendar misses some events

[Fjodor42]

⚠️ This issue respects the following points: ⚠️

• [X] This is a bug, not a question or a configuration/webserver/proxy issue.
• [X] This issue is not already reported on Github (I've searched it).
• [X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• [X] Nextcloud Server is running on 64bit capable CPU, PHP and OS.
• [X] I agree to follow Nextcloud's Code of Conduct.

Bug description

For a number of years, I have run my own NextCloud instance, on which I have renamed my user's default calendar to "Fælles Kalender" (Danish for Common Calendar), so my wife and I can both enter events into it, to update each other on events that affect us both.

Both of us sync from Android phones (via DAVX5 as recommended), and I have also been subscribed via Thunderbird on both my personal and my work computers.

As for the latter subscriptions, I used to just use the built-in CalDAV functionality, but have switched to the CalDAV connector for TbSync as per recommended, but to no avail of the problem described below.

For quite some time, we have experienced the unfortunate state of affairs that, seemingly at random, her updates from her Android phone will not show up in my sync'ed applications, vice versa, and sometimes previously sync'ed events seem to "go missing".

Due to lack of mental and chronological surplus, I have postponed looking further into this, but it cannot continue.

I have recently noticed that the events will typically be shown in the web view of the calendar, and will, typically, show up if I un- and resubscribe, leading me to believe that the problem does not lie in actually storing new events or alterations to existing events, but I am at a loss regarding how to even ascertain whether this is even a NextCloud problem, or a problem shared between CalDAV implementations of Thunderbird, TbSync and DAVX5.

Hence, initially, I am asking for help pinpointing where the problem lies, but have labeled is a bug, as the common denominator seem to be NextCloud, rather than the various clients.

Please advise

Steps to reproduce

1. Have more than one user share a calendar
2. Have them sync with multiple clients
3. ???
4. Events disappear

Expected behavior

Shared access to a calendar should mean that all participants, whatever their CalDAV client, should be able to create and alter events and have their events, or changes to such, be reflected to all participants, regardless of the CalDAV-compliant client software used to sync.

Installation method

Community Web installer on a VPS or web space

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.0

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• [X] Default user-backend (database)
• [ ] LDAP/ Active Directory
• [ ] SSO - SAML
• [ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "hashingCost": 10,
        "trusted_domains": [
            "nephele.molgaard.org"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/nephele.molgaard.org",
        "version": "25.0.3.2",
        "dbtype": "pgsql",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "logtimezone": "UTC",
        "installed": true,
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "filelocking.enabled": true,
        "memcache.local": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "timeout": 0
        },
        "maintenance": false,
        "loglevel": 2,
        "logfile": "\/var\/log\/nextcloud\/nextcloud.log",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwritehost": "nephele.molgaard.org",
        "overwriteprotocol": "https",
        "overwritecondaddr": "^195\\.201\\.133\\.130$",
        "enable_previews": true,
        "updatechecker": true,
        "htaccess.RewriteBase": "\/",
        "theme": "",
        "updater.release.channel": "stable",
        "app_install_overwrite": [
            "files_downloadactivity",
            "ojsxc",
            "calendar",
            "contacts",
            "spreed",
            "socialsharing_twitter",
            "bruteforcesettings",
            "socialsharing_facebook",
            "files_trackdownloads"
        ],
        "default_phone_region": "ISO 3166-2:DK",
        "secret": "***REMOVED SENSITIVE VALUE***"
    }
}

List of activated Apps

Enabled:
  - activity: 2.17.0
  - admin_audit: 1.15.0
  - bruteforcesettings: 2.5.0
  - calendar: 4.2.3
  - circles: 25.0.0
  - cloud_federation_api: 1.8.0
  - comments: 1.15.0
  - contacts: 5.1.0
  - contactsinteraction: 1.6.0
  - dashboard: 7.5.0
  - dav: 1.24.0
  - deck: 1.8.3
  - federatedfilesharing: 1.15.0
  - federation: 1.15.0
  - files: 1.20.1
  - files_downloadactivity: 1.15.0
  - files_external: 1.17.0
  - files_pdfviewer: 2.6.0
  - files_rightclick: 1.4.0
  - files_sharing: 1.17.0
  - files_trackdownloads: 1.11.0
  - files_trashbin: 1.15.0
  - files_versions: 1.18.0
  - firstrunwizard: 2.14.0
  - logreader: 2.10.0
  - lookup_server_connector: 1.13.0
  - nextcloud_announcements: 1.14.0
  - notifications: 2.13.1
  - oauth2: 1.13.0
  - password_policy: 1.15.0
  - photos: 2.0.1
  - privacy: 1.9.0
  - provisioning_api: 1.15.0
  - recommendations: 1.4.0
  - related_resources: 1.0.4
  - serverinfo: 1.15.0
  - settings: 1.7.0
  - sharebymail: 1.15.0
  - socialsharing_facebook: 2.5.0
  - socialsharing_twitter: 2.5.0
  - support: 1.8.0
  - survey_client: 1.13.0
  - systemtags: 1.15.0
  - text: 3.6.0
  - theming: 2.0.1
  - twofactor_backupcodes: 1.14.0
  - updatenotification: 1.15.0
  - user_status: 1.5.0
  - viewer: 1.9.0
  - weather_status: 1.5.0
  - workflowengine: 2.7.0
Disabled:
  - encryption: 2.10.0
  - ojsxc: 5.0.0
  - spreed: 15.0.3
  - suspicious_login: 4.2.1
  - twofactor_totp
  - user_ldap

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Irrelevant for now, I think, as I have not myself been able to find anything in the logs.

Please do advise as to changes to logging levels if relevant.

Additional info

No response

[gohrner]

Currently, Thunderbird will randomly hide / not show events from Nextcloud calendars.

I've no idea if it's a Thunderbird or a Nextcloud bug.

I reported that a while ago in the Thunderbird Bugzilla, unfortunately with no official reaction so far:

https://bugzilla.mozilla.org/show_bug.cgi?id=1795720

Currently, Nextcloud calendars in Thunderbird are basically unusable due to this. (Without exaggaration, you just never now if you have an appointment or not if you check the calendar view in Thunderbird.)

[Fjodor42]

I will add that this holds both for syncing directly with Thunderbird, via TbSync and on our phones with DAVX5 - whether or not this is a result of Thunderbird shenanigans, though, I cannot say, but it might not be the case.

[gohrner]

@Fjodor42: Mh, I'm also using DAVx⁵ on my Android with the same Nextcloud calendars, and the Android calendar is flawlessly synced with Nextcloud.

It's only Thunderbird (using its built-in CalDAV support, I didn't try TBSync) which randomly hides many appointments / events in the synced calendars.

Can you describe exactly under which circumstances you're missing events / calendars on your phone?

[Fjodor42]

Unfortunately, I cannot.

There seems to be a bias to what my wife puts into the calendar going missing, but this may very well just be a function of her being the one who puts in the most new events.

And, for the record, while Thunderbird has been set up on her computer, she mostly uses her phone anyway, which means DAVX5.

So, to recap the broad outline, it's DAVX5 on two devices, Thunderbird on 2 (three including my wife's), and a haze of hit-and-miss as to whether events stick, leading me to speculate that Nextcloud might also have a problem, but your trouble with Thunderbird certainly does lend credence to theories about that being the/a culprit.

[OliverBreuer]

I have a similar, probably the same problem:

In the last four weeks I had twice the problem that some calendar entries on shared calendars were not synced to phones.

Here is the information I have:

Setup:

• self-hosted nextcloud (current version)
• 3 android phones
• syncing on the phones via davx5

first problem (about two weeks ago):

• about 3-5 entries (probably more) were synced to one phone but not to the other two
• new entries were synced normally
• unsyncing the calendar on the phones and resyncing finally synced the missing entries

second problem (today):

• 4 entries (probably more) were entered in the nextcloud calendar via browser
• none of the three phones synced these entries
• new entries were synced normally
• exporting (in the nextcloud web app) the not-synced and a new synced entry showed no difference in the vcal-data other than in the expected properties (CREATED, DTSTAMP, LAST-MODIFIED, UID, DTSTART, DTEND, SUMMARY)
• changing one of the not-synced entries via browser resulted in syncing this entry to the phones
• unsyncing the calendar on the phones and resyncing finally synced all the missing entries

In both cases it looks like the entries weren't transferred in one incremental update. Of course, in the next incremental updates they weren't send because they didn't change. After changing one of the entries it was synced because it was included in the next incremental update.

Unfortunately I have no log of the original sync. So I don't now whether the first incremental update, where the entries should have been included, just missed these entries or whether this update was somehow aborted or whether something else went wrong.

If anyone has any idea what to try to reproduce this issue, I'm happy helping testing and gathering logs.

[gohrner]

@OliverBreuer: That's different from what I encounter - I'd assume it's a different problem and should be reported as a different bug (though not sure whether it's a Nextcloud bug or DavX5 then):

• I also use DavX5 and have not detected even one missing event.
• In my Thunderbird calendar, also events which were displayed suddenly disappear. It's also not only few events that disappear, but over the course of a day actually most of them for that day.

So there's not single eventy missing from the calendar, but basically the other way around - in the end, only a few events still show up in Thunderbird.

Maybe there's the same underlying cause, but I'm not sure... The observation described in the original bug description above more seems to match what I'm observing and less what you're experiencing...

[Fjodor42]

@gohrner , @OliverBreuer: As for the original bug description, being written by me, I would like to state that both of your observations seem to fit within what I tried to describe.

[flortsch]

I am experiencing the same issue across various clients (macOS/iOS, Thunderbird/Evolution, DavX5) since I upgraded to Nextcloud 26.0.1.

A client creates an event and this is synced to the Nextcloud server (the event appears in the Nextcloud calendar app). But the entries sometimes don't appear on some clients, even if I force refresh the calendar on the client. I can't tell whether the new entries are not synced to the clients or synced but simply not shown due to some other issue.

The issue occurs on shared calendars with different users and clients, but also on a single calendar with the same user and different clients. Unfortunately, I can't reproduce the issue reliably. Any guidance on how to debug this issue is highly welcome.

[Eskawotz]

I have the same problem as Flortsch since upgrading to Nextcloud 26. I too then reinstalled various instances from scratch, but the problem remains the same. I was able to figure out that only entries are not synced with another client if the creation of the entry is older than 12 hours. It ALWAYS works if I, for example, create a new entry with the iPhone and start Thunderbird right after or Thunderbird is running during the creation. So the problem occurs if I e.g. create an entry with the iPhone and Thunderbird is not running on Windows/Linux and start it12 hours later. Then this entry is not synchronized, although I can see it in the Nextcloud Calendar app.

[ckroenicke]

We have the same problem as flortsch too. Nextcloud 26 with shared calendars synced with various clients (Thunderbird, DAVx5). Sometimes an event created on one client is correctly synced to Nextcloud but then only to some of the other clients. Or an event that was correctly synced to all clients gets changed on one of the clients but the change will not appear on some of the others. I don't think we experienced disappearing events though.

@Eskawotz: I'll check whether I can reproduce the bug when waiting 12 hours until I reopen my Thunderbird after I created a new event/updated an existing one on another client. I'll report back what happend.

[Eskawotz]

One more comment on that bug: After upgrading from nextcloud25 to nextcloud26 it was very fuzzy with that error, as mobile devices are always online and do the sync regularly which doesn't run into that error. I guess that is the reason why so less info can be found regarding this issue.

But now I figured out something out, which makes it even more weird: I deleted the account in Thunderbird and added it again and voila: All events have being synced!

But the general issue remains.

I made a rollback to Nextcloud25, cause this is such a nasty issue.

[ckroenicke]

Here the result of my test: I closed my Thunderbird so it could not syncronize and then added an event and updated an existing one. Then I waited more then 24 hours before starting Thunderbird and syncronizing the calendar.

I then started to look into the source code to get an idea what change could have caused that behavior. I found out that last october a background job was added that once a day prunes the list of sync tokens which are used to identify the (calendar) changes that need to be synced (see https://github.com/nextcloud/server/commit/6f158733211749ca6d70858ee58ba35e85d46e81). Maybe this has something to do with the strange behavior?

[flortsch]

I also thought that this prune job could be related to this issue, but the job should keep an amount of change objects in the db (10k per default) when pruning (which should be more than enough for small instances). You could test if disabling cronjobs fixes your issue.

I have upgraded my instance to 26.0.2 and will report if I still face this issue. Also will check out if this happens after the 12h of not syncing clients.

[charno]

Thanks for analyzing! This bug drives me crazy since forever, and I always suspected thunderbird since no events are missing on my mobile. But the mobile syncs every hour, so...

@Eskawotz : An easier workaround (instead of deleting / readding) is to disable offline support in the thunderbird calendar, save, re-enable, save. Then all events are re-synced as well.

@ckroenicke @flortsch If I understand your comments correctly, then at least the last 10k change objects should still be present in the table oc_calendarchanges? That's clearly not the case in my installation... (And my calendar is huge, with daily changes since years, check the ID). The only change that is present is from today (the URI seems to contain a unix timestamp from today). So maybe there is a bug in this prune job which prevents keeping the last 10k changes?

MariaDB [nextcloud]> select * from oc_calendarchanges;
+-------+---------------------+-----------+------------+-----------+--------------+
| id    | uri                 | synctoken | calendarid | operation | calendartype |
+-------+---------------------+-----------+------------+-----------+--------------+
| 35573 | 1685887586.R976.ics |     19950 |          2 |         1 |            0 |
+-------+---------------------+-----------+------------+-----------+--------------+
1 row in set (0.001 sec)

[Eskawotz]

I have also upgraded to 26.0.2 (another test-instance, productive is 25!) and it still has that issue.

@charno: Thank you for that hint! Just to understand: @flortsch: you would recommend to stop the cron-job? I can try and check whether the issue remains! My test instance is only one admin and a user with which this problem occurs. I doubt this entry has to do with it (10000 is a huge number)

[flortsch]

Okay, so I tested the prune job on my instance. I updated a calendar event on my Evolution client, synced it with Nextcloud which resulted in a new row in the oc_calendarchanges table and manually triggered the prune job (I did this by reducing the time interval in apps/dav/lib/BackgroundJob/PruneOutdatedSyncTokensJob.php to a few seconds instead of one day and run php cron.php to trigger the job).

Indeed, the job did not keep the row and cleared the whole table! Looking at the code it seems to me that an SQL delete statement with a offset parameter is created that might not be supported on all DB types (running MariaDB here).

However, I don't fully understand the impact of these missing rows in the oc_calendarchanges table and whether this could lead to missing event updates. Because I fired up my second client (Android Dav5x) after I executed the prune job that cleared my table, and the Dav5x client still synced the event update (maybe pressing the sync button in Dav5x triggers a full re-sync?).

[flortsch]

@Eskawotz Yeah, it seems to me that the prune job code is buggy. It did not keep any rows and wiped my oc_calendarchanges table. However, my second client (Dav5x) still synced my updated calendar event (maybe pressing the sync button in Dav5x triggers a full re-sync?). But it might be worth disabling this job and checking if this fixes sync issues on other types of clients (like iOS/MacOS, Thunderbird, etc.).

[flortsch]

The easiest way to disable this job without globally disabling cron for me seems to just comment all the code in the run function of apps/dav/lib/BackgroundJob/PruneOutdatedSyncTokensJob.php.

        public function run($argument) {
//              $limit = max(1, (int) $this->config->getAppValue(Application::APP_ID, 'totalNumberOfSyncTokensToKeep', '10000'));
//
//              $prunedCalendarSyncTokens = $this->calDavBackend->pruneOutdatedSyncTokens($limit);
//              $prunedAddressBookSyncTokens = $this->cardDavBackend->pruneOutdatedSyncTokens($limit);
//
//              $this->logger->info('Pruned {calendarSyncTokensNumber} calendar sync tokens and {addressBooksSyncTokensNumber} address book sync tokens', [
//                      'calendarSyncTokensNumber' => $prunedCalendarSyncTokens,
//                      'addressBooksSyncTokensNumber' => $prunedAddressBookSyncTokens
//              ]);
        }

[ckroenicke]

I forgot to add that we also use Nextcloud (26.0.2) with a MariaDB (10.3.32). We disabled the code in PruneOutdatedSyncTokensJob.php as suggested by @flortsch and will see tomorrow whether that fixes the issue for the moment.

[Eskawotz]

I have commented out as well and will check.. I will let you know! Thank you.....

[flortsch]

For completeness, I digged deeper into the code, especially CalDavBackend.php#getChangesForCalendar. Missing rows in oc_calendarchanges will definitely affect clients that request calendar changes with a given sync token. They simply won't get any updates. I also enabled SQL debugging in my MariaDB database to see what SQL statement the prune job executes, it simply is DELETE FROM oc_calendarchanges, i.e., it will delete all rows in the table no matter what. Since the prune job also calls a prune routine in the CardDavBackend that uses a similar DELETE statement, this should also be an issue when synchronizing contact updates. I will report both in a new issue and maybe also provide a fix for the SQL statement.

[charno]

@flortsch I'm also on it, already have the dev environment setup.

The vscode devcontainer setup uses postgre, and the error is present there as well.

[charno]

Hmm... Doesn't seem to easy a bug:

    private function getSQLForDelete(): string
    {
        $table = $this->sqlParts['from']['table']
            . ($this->sqlParts['from']['alias'] ? ' ' . $this->sqlParts['from']['alias'] : '');

        return 'DELETE FROM ' . $table
            . ($this->sqlParts['where'] !== null ? ' WHERE ' . ((string) $this->sqlParts['where']) : '');
    }

vs.

    private function getSQLForSelect(): string
    {
        $query = 'SELECT ' . ($this->sqlParts['distinct'] ? 'DISTINCT ' : '') .
                  implode(', ', $this->sqlParts['select']);

        $query .= ($this->sqlParts['from'] ? ' FROM ' . implode(', ', $this->getFromClauses()) : '')
            . ($this->sqlParts['where'] !== null ? ' WHERE ' . ((string) $this->sqlParts['where']) : '')
            . ($this->sqlParts['groupBy'] ? ' GROUP BY ' . implode(', ', $this->sqlParts['groupBy']) : '')
            . ($this->sqlParts['having'] !== null ? ' HAVING ' . ((string) $this->sqlParts['having']) : '')
            . ($this->sqlParts['orderBy'] ? ' ORDER BY ' . implode(', ', $this->sqlParts['orderBy']) : '');

        if ($this->isLimitQuery()) {
            return $this->connection->getDatabasePlatform()->modifyLimitQuery(
                $query,
                $this->maxResults,
                $this->firstResult
            );
        }

        return $query;
    }

The query builder seems to not honor setFirstResult for delete queries... And the query builder is external (doctrine/dbal)

[charno]

https://github.com/charno/server/commit/9f1eb49196087b50f06f60442c3e730e220c1361

Failing test case added

[charno]

Pull request https://github.com/nextcloud/server/pull/38639 opened. Hoping for fast merge :-)

[flortsch]

Nice! Thanks @charno for fixing the code and creating the PR :) Hoping for fast merge!

[ckroenicke]

@charno Thanks for fixing from me too! But could you improve the test for the pruning a little bit by checking whether the newest change is left after pruning? It would be good to verify that the correct changes were deleted... ;)

[gohrner]

I (very) recently noticed trouble syncing with DavX5 on Android which I had not seen before. It matches with some of the descriptions here, but probably is a separate issue from the "disappearing" events in Thunderbird (which seem to be synced, but are "simply" not shown in the Calendar view under somewhat "random" (but frequent) circumstances...)

A new issue I noticed in the last few days is an increasing number of 400 errors in DavX5 when syncing / accepting / removing calendar invitations through mails. :-/

The only thing which changed server-side was an upgrade of the NC calendar app (but not the server itself), so maybe that's more caused by DavX5 upgrades, of which there were several recently?

I'll continue to observe this... It renders my calendars even more unusable than before (now, all three calendar views always disagree, while until about 2 weeks ago, at least the web view and the Android calendar were in sync, with only Thunderbird "losing" calendar entries), but I unfortunately don't have time a the moment to investigate this more in depth. :-(

I'm curious to see if @charno's fixes will improve the situation.

[charno]

@gohrner For thunderbird it's also the case that some events are not synchronized. It's not only a display error on thunderbirds side.

If you have your own nextcloud server, you can try to apply the patches. It's only two files that need very local editing: https://github.com/charno/server/commit/73fb2997f4b4e4855ef457aae8fde9cda0839e22 :-)

[gohrner]

@charno: Mh, I'm still on Nextcloud 25 and it does not even seem to have a pruneOutdatedSyncTokens method in these classes...

So the problems I'm currently experiencing seem to stem from something else. :-(

However, this also means that probably the original bug report is unrelated to your fix (or the other way around), as your fix then repairs a newly introduced breakage which is more recent than this bug report.

BTW, if someone on NC 26 wants to test @charno's patch, you can download it from GitHub as follows: https://github.com/charno/server/commit/73fb2997f4b4e4855ef457aae8fde9cda0839e22.diff

That's a GitHub feature I didn't know and just found with a Google search. Just append .diff to a commit view to export it as a unified diff.