Certain details (e.g. addresses) are not syncing between phone and server

[hansdez]

trafficstars

Steps to reproduce

1. Have your contacts synched between Nextcloud Server and your Android phone using DAVx5 (I use 'Version 3.1.1-gplay (301010002)'). Make sure that if you create a new contact on the server that it appears on your phone after a sync.
2. On the server (through the web interface) change the address of a contact and make sure that it is saved.

Expected behaviour

It is expected that the change is propagated to the phone after a sync by DAVx5.

Actual behaviour

The change is not propagated to the phone. There is now a situation where the address on the server and the address on the phone are different for a particular contact. Even after a sync.

Server configuration detail

Operating system: Linux 4.15.0-106-generic #107-Ubuntu SMP Thu Jun 4 11:27:52 UTC 2020 x86_64

Webserver: nginx/1.14.0 (fpm-fcgi)

Database: mysql 5.7.30

PHP version:

7.2.24-0ubuntu0.18.04.6 Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, Reflection, SPL, sodium, session, standard, cgi-fcgi, mysqlnd, PDO, xml, apcu, bz2, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, iconv, imagick, intl, json, exif, mysqli, pdo_mysql, pdo_sqlite, Phar, posix, readline, shmop, SimpleXML, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 18.0.6 - 18.0.6.0

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: unknown

Signing status

Array ( )

List of activated apps

Enabled:
 - activity: 2.11.0
 - bruteforcesettings: 1.6.0
 - calendar: 2.0.3
 - cloud_federation_api: 1.1.0
 - contacts: 3.3.0
 - dav: 1.14.0
 - deck: 1.0.2
 - federatedfilesharing: 1.8.0
 - files: 1.13.1
 - files_pdfviewer: 1.7.0
 - files_rightclick: 0.15.2
 - files_sharing: 1.10.1
 - files_trashbin: 1.8.0
 - files_versions: 1.11.0
 - files_videoplayer: 1.7.0
 - issuetemplate: 0.6.0
 - logreader: 2.3.0
 - lookup_server_connector: 1.6.0
 - nextcloud_announcements: 1.7.0
 - notes: 3.5.1
 - notifications: 2.6.0
 - oauth2: 1.6.0
 - password_policy: 1.8.0
 - photos: 1.0.0
 - privacy: 1.2.0
 - provisioning_api: 1.8.0
 - serverinfo: 1.8.0
 - settings: 1.0.0
 - sharebymail: 1.8.0
 - text: 2.0.0
 - theming: 1.9.0
 - twofactor_backupcodes: 1.7.0
 - twofactor_totp: 4.1.3
 - twofactor_u2f: 5.1.0
 - updatenotification: 1.8.0
 - viewer: 1.2.0
 - workflowengine: 2.0.0
Disabled:
 - accessibility
 - admin_audit
 - comments
 - encryption
 - federation
 - files_external
 - firstrunwizard
 - recommendations
 - support
 - survey_client
 - systemtags
 - user_ldap

Configuration (config/config.php)

{
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "files.enhancedstrategy.com"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "dbtype": "mysql",
    "version": "18.0.6.0",
    "overwrite.cli.url": "https:\/\/files.enhancedstrategy.com",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "mysql.utf8mb4": true,
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "memcache.local": "\\OC\\Memcache\\APCu",
    "mail_smtpmode": "smtp",
    "mail_smtpsecure": "tls",
    "mail_sendmailmode": "smtp",
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpauthtype": "LOGIN",
    "mail_smtpauth": 1,
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpport": "587",
    "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
    "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
    "twofactor_enforced": "true",
    "twofactor_enforced_groups": [],
    "twofactor_enforced_excluded_groups": [],
    "maintenance": false,
    "simpleSignUpLink.shown": false,
    "loglevel": 2
}

Are you using external storage, if yes which one: No

Are you using encryption: No

Are you using an external user-backend, if yes which one: No

Client configuration

Browser: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0

Operating system:

Logs

Web server error log

n/a

Nextcloud log

n/a

Browser log

n/a

[skjnldsv]

cc @rfc2822 :) How can we help debugging this?

(And hope you're doing ok over there, long time no see! :wave: :hugs:)

[rfc2822]

Sounds like an inconsistency between LABEL and ADR.

What are "certain details (e.g. addresses)"? The report is only about addresses. What are the other details? Are there steps to reproduce the problem?

DAVx5 debug logs (verbose logging) should contain all information required to debug that.

[hansdez]

Apologies for the lack of clarity. I am now using a regular version of Android and am really disliking Google's contacts app which does all sort of things without me necessarily wanting it (or being clear about it).

The report is about addresses, because I know for sure they were giving a problem. But I also had a problem with a name update, but didn't dig further into it.

Both the name field and the address field share this (to me) weird phenomenon where you can either store the data in a single field (i.e. firstname and surname in one field) or you can store it in multiple fields (i.e. by filling in a detailed name as Nextcloud calls it). Google wants to store an address in a single field as a string (and autocompletes it for you in their contacts app), whereas Nextcloud prefers to spread an address out over multiple fields.

I've done some extra testing with verbose logging to help with the debugging. What I have done is the following:

• Checked if there was an update for DAVx5 (there wasn't, so am using version '3.2.1.1-gplay (302010100)').
• Set the sync frequency of the 'Contacts sync. interval' of my Nextcloud account in DAVx5 to 'Only manually', and (for privacy reasons) turnen off the syncing of my calendars.
• Turned on verbose logging
• Created a new contact on the Nextcloud server using the standard web interface and filled quite a few details:

• I then did a manual sync via DAVx5. The contact came through as expected.
• I next changed all the details of the contact on the server side:

• I synced DAVx5 and The changes came through on my phone as expected.
• I then only changed the address on the server side and synced. This worked fine too.
• I then changed the contact on my phone and used Google's way to find an address. Google stored this in a single field.
• I synced with DAVx5 and the results came through on the server side:

• There is one small problem though with Nextcloud (I guess?) adding a slash right after the city and before the comma (or maybe forgetting to strip a slash). Do let me know if I should create a separate issue for this.

• I also got this message ‘This contact was broken and received a fix. Please review the content and click here to save it.’ Not sure why that happens and not the most usefu message, but I clicked.
• I then only changed the address on the server side:

• After a DAVx5 sync I could not see the change on my phone:

• However when I clicked edit, I could see that some of the changes did come through although they were not shown when displaying the contact (city and Zip Code), but not the street:

• I saved the DAVx5 log.

I am not comfortable with sharing my logfiles here on GitHub (for eternity), but you can download a ZIP-file with 'davx5-log.txt' and 'davx5-info.txt' via the following URL (within a week):

Removed as the logs have already been used

Does this help you in seeing where the problem may be? Please tell me if you need anything else. I am not a programmer, just an enthusiastic user, so not very fluent in how things are done here on GitHub.

[rfc2822]

Thanks for the clarification and the logs! The DAVx5 logs show this vCard which is received by DAVx5 when the first problem (NewAddress2AfterChangeOnPhone not coming through) occurs:

…
ADR;TYPE=work;LABEL="Haarlemmerplein 10, 1013 HS Amsterdam, Netherlands":;;
 NewAdressAfterChangeOnPhone 2;NewCityAfterChangeOnPhone;;3333 GG;
…

As you can see, the LABEL is still "Haarlemmerplein 10…", which is then inserted into the Android database as FORMATTED_ADDRESS (data1):

2020-07-16 13:02:25 81797 [AndroidContact] Built StructuredPostal data row PARAMETER #0 = mType: 1, mUri: content://com.android.contacts/data?caller_is_syncadapter=true&account_name=Contacts%20(nextcloud%40domain.tld%20nA)&account_type=at.bitfire.davdroid.address_book, mSelection: null, mExpectedCount: null, mYieldAllowed: false, mValues: raw_contact_id=2462 data9=3333 GG data8=null data7=NewCityAfterChangeOnPhone data6=null data5=null data4=NewAdressAfterChangeOnPhone 2 data3=null data2=2 data1=Haarlemmerplein 10, 1013 HS Amsterdam, Netherlands data10=null mimetype=vnd.android.cursor.item/postal-address_v2, mValuesBackReferences: null, mSelectionArgsBackReferences: null

I think that all problems/misunderstandings have their origin in the diverging contents of ADR and its LABEL. Maybe Nextcloud Contacts could

1. either update/regenerate the LABEL parameter on address updates or
2. drop the LABEL, if it can't understand/update it; and
3. maybe show the LABEL as an extra field? Something like "Formatted address" (should be an optional multi-line field).

Or maybe DAVx5 should send no LABEL at all? However, Android has this FORMATTED_ADDRESS field, and I have mapped it to the adr-label, because for me it seems to be useful for this purpose.

[hansdez]

Would it be possible for you take out my email address from the quotes log file (i.e. my account name)?

[rfc2822]

of course

[creopard]

Experienced the same behaviour while dealing with #1749

[bugariu]

Hi, this bug is very annoying. Would be nice to get it solved somehow...

Thanx!

[oncer]

I have experienced the same issue. When I edit the address of a contact using the web interface, it is not synced to my Android phone. Nextcloud Contacts 3.4.2, DAVx5 3.3.8-ose. Is there a workaround maybe? Edit: When I add a second address it gets synced properly. So my workaround for now is to not edit address fields but add the modified address as a new field. Then delete the old address later.

[timtester-boop]

Experiencing the same issue. Other fields except the address are synchronized properly. I'm using the same workaround as oncer described. Is there a fix to this problem?

[Parsnip]

Found this issue when I was looking for a way to disable the automatic LABEL creation that dav5x does. I basically have apparently an unholy mix of runbox carddav that I'm trying to use on the runbox web interface, Google Contacts and Thunderbird. All 3 appear to have different ideas on what to do and what not do with the LABEL, and as far I can see if I could disable the LABEL creation altogether, my problem might go away.

EDIT: Actually I can't be sure that Google Contacts isn't also doing the same thing, so even if davx5 had the option, I can't be sure it would actually solve my issue.

EDIT2: I also realize that my issue isn't a nextcloud issue at all, but rather davx5 is the commonality here. It's unfortunate to see that this has been a problem for over a year without a solution.

[Redsandro]

I confirm this problem, or at least part of it. I don't know about the technical details above that cause it, but let me explain my observations:

1. Android/LineageOS with DAVx⁵ contact syncing with NextCloud 20 works in both directions as expected. :heavy_check_mark:
2. Upgrade NextCloud 20 > 21 > 22 :heavy_check_mark:
3. Contact address changes only sync from Android to NC22. Not the other way around. :warning:

Something changed between NC20 and NC22 that - from a user point of view - broke something that used to work.

I've been having this problem for a while now, and I assumed it was "in your face" enough for it to get noticed and fixed. I guess I got impatient and searched the issues. So even though NC introduced a breaking change, perhaps not many people are using DAVx⁵ with NC?

[rfc2822]

At the moment I tend to disable LABEL generation of DAVx⁵ at all. There are almost no other clients that parse/use it correctly, its syntax is different in vCard3/vCard4 (the next source of confusion) and while it's a nice feature in theory, it breaks address sync in reality. Being able to exchange the formatted address in addition to the structured address would be nice, but at the moment, on both Android and CalDAV side one can be happy if even structured addresses are working…

[Redsandro]

At the moment I tend to disable LABEL generation of DAVx⁵ at all.

Where did you find this option in DAVx⁵? All I see for CardDAV is the "Groups are separate vCards" option.

[rfc2822]

At the moment I tend to disable LABEL generation of DAVx⁵ at all.

Where did you find this option in DAVx⁵? All I see for CardDAV is the "Groups are separate vCards" option.

There is no option. I tend to disable it in the source code for future releases (I'm a DAVx5 developer). Maybe we could also make an option.... but I strongly dislike options

[Redsandro]

I tend to disable it in the source code for future releases (I'm a DAVx5 developer).

Ah I see. You intend to disable the DAVx⁵ feature/field that NC Contacts doesn't interpret correctly. Well that certainly sounds good. Admittedly I'm not fully aware of the consequences.

[rfc2822]

Yes, almost no client, neither Android can deal with the field correctly...

[rfc2822]

Further versions of DAVx⁵ won't send the LABEL anymore (not only of this issue, but because formatted addresses don't work well on Android anway), which should also avoid this problem here when DAVx⁵ is used (however not with other CalDAV clients which still may send a LABEL).

If anyone is interested in testing, please let me know.