Radicale radicale upload VCARD2.1 failed

In debian11, apt install radicate installed radicate-3.0.6.

when I upload a VCF file via browser, it failed.

This is the VCF file:

BEGIN:VCARD
VERSION:2.1
N;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:;=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=
=95=E6=B5=8B=E8=AF=95=E6=B5=8B;;;
FN;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=
=95=E6=B5=8B=E8=AF=95=E6=B5=8B
TEL;CELL:12 345 678 90
PHOTO;ENCODING=BASE64;JPEG:/9j/4AAQS....
 2wBDAQcHBwo....

END:VCARD

I changed VCF file, then upload OK.

change QUOTED-PRINTABLE in one line.
change PHOTO:ENCODING=BASE64; to PHOTO:ENCODING=B;

BEGIN:VCARD
VERSION:2.1
N;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:;=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B;;;
FN;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B=E8=AF=95=E6=B5=8B
TEL;CELL:12 345 678 90
PHOTO;ENCODING=B;JPEG:/9j/4AAQS....
 2wBDAQcHBwo....

END:VCARD

Apr 25 '22 07:04 osnosn

Same problem here, exported contacts, tried to upload it to radicale and fails with 400 Bad Request

Debian 11, Nginx, HTTP2, Radicale 3.1.5

Jul 11 '22 19:07 Fijxu

I can confirm this is still happening on 3.1.7.0, Running Docker on Unraid.

Oct 13 '22 01:10 quamok

I've been trying to upload my contacts (vCard 3.0) and I get the same thing.

[edit] Since there's been no activity on this project since July 14, I guess this isn't the solution I was looking for.

Nov 10 '22 20:11 Barugon

I was able to fix this by commenting the following lines from the vobject package vcard.py file

    def decode(cls, line):
        """
        Remove backslash escaping from line.valueDecode line, either to remove
        backslash espacing, or to decode base64 encoding. The content line should
        contain a ENCODING=b for base64 encoding, but Apple Addressbook seems to
        export a singleton parameter of 'BASE64', which does not match the 3.0
        vCard spec. If we encouter that, then we transform the parameter to
        ENCODING=b
        """
        if line.encoded:
            if 'BASE64' in line.singletonparams:
                line.singletonparams.remove('BASE64')
                line.encoding_param = cls.base64string
            encoding = getattr(line, 'encoding_param', None)
            if encoding:
                if isinstance(line.value, bytes):
                    line.value = codecs.decode(line.value, "base64")
#                else:
#                    line.value = codecs.decode(line.value.encode("utf-8"), "base64")
            else:
                line.value = stringToTextValues(line.value)[0]
            line.encoded = False

The encoding gets all wonky when BASE64 is used for the encoding param. I am not sure why and I am sure this quick fix breaks other cases, but it allows me to use base64 encoding on photos now so that's all that matters for me.

Dec 15 '22 04:12 slyduda

"radicale" depends in this case 100% on vobject -> one has to wait until "vobject" got proper fixed

Dec 15 '22 06:12 pbiering

I have constructed an example vCard, using the first quoted template above, and current vObject parses it without any problem.

The code segment above is a little dubious, in several ways, but ... for data that is properly BASE64 encoded, the encoding of the string value to bytes will change nothing (since BASE64 is pure ASCII, and pure ASCII encoded as UTF-8 is byte-wise identical).

It'd be great if one of the reporters (@osnosn, @Fijxu, @quamok, @slyduda) could provide the Radicale log file showing one of these failed imports -- it'd be really useful to see exactly how it's failing.

Mar 02 '24 22:03 da4089

I have constructed an example vCard, using the first quoted template above, and current vObject parses it without any problem.

The code segment above is a little dubious, in several ways, but ... for data that is properly BASE64 encoded, the encoding of the string value to bytes will change nothing (since BASE64 is pure ASCII, and pure ASCII encoded as UTF-8 is byte-wise identical).

It'd be great if one of the reporters (@osnosn, @Fijxu, @quamok, @slyduda) could provide the Radicale log file showing one of these failed imports -- it'd be really useful to see exactly how it's failing.

Traceback (most recent call last):
  File "/home/ytf/ws/py_ws/Radicale/radicale/app/put.py", line 145, in do_PUT
    vobject_items = radicale_item.read_components(content or "")
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ytf/ws/py_ws/Radicale/radicale/item/__init__.py", line 52, in read_components
    return list(vobject.readComponents(s))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ytf/anaconda3/envs/radi/lib/python3.12/site-packages/vobject/base.py", line 1101, in readComponents
    vline = textLineToContentLine(line, n)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ytf/anaconda3/envs/radi/lib/python3.12/site-packages/vobject/base.py", line 925, in textLineToContentLine
    return ContentLine(*parseLine(text, n), **{'encoded': True,
                        ^^^^^^^^^^^^^^^^^^
  File "/home/ytf/anaconda3/envs/radi/lib/python3.12/site-packages/vobject/base.py", line 813, in parseLine
    raise ParseError("Failed to parse line: {0!s}".format(line), lineNumber)
vobject.base.ParseError: At line 159: Failed to parse line: =30=34=33=33=3F=E9=97=AA=E9=97=AA=48=E7=BA=A2=E6=98=9F
[2024-04-25 13:20:54 +0800] [141563/Thread-18 (process_request_thread)] [DEBUG] Response content:
Bad Request
[2024-04-25 13:20:54 +0800] [141563/Thread-18 (process_request_thread)] [INFO] PUT response status for '/ytf//f17655c9-2159-bae2-e297-cd403db8cb31/' in 0.090 seconds: 400 Bad Request

Apr 25 '24 05:04 yangtfu

Is this special vcard passing the validator without any errors? Please check with https://icalendar.org/validator.html But even if it's the case, as one see, it's related to vobject.

Apr 25 '24 05:04 pbiering

Thanks for the further info, @yangtfu!

The line that reports a parsing problem:

"=30=34=33=33=3F=E9=97=AA=E9=97=AA=48=E7=BA=A2=E6=98=9F"

run through base64.b64decode(), ends up being:

b'\xdfM\xf8\xdf}\xf7\xdcQ=\xf7\xb0\x00\x13\xdf{\x00\x0e<\x13\xb0@\x03a:\xf7\xcfE'

which ... doesn't look like it'd be a valid vCard line. It's not valid UTF-8 either. So I'm not sure what it's trying to do.

Could you please copy the whole problematic vCard into a comment? Maybe it's a continuation line or something ... With the complete input, hopefully we can figure this out.

Apr 25 '24 08:04 da4089

Thanks for the further info, @yangtfu!

The line that reports a parsing problem:
"=30=34=33=33=3F=E9=97=AA=E9=97=AA=48=E7=BA=A2=E6=98=9F"
run through base64.b64decode(), ends up being:
b'\xdfM\xf8\xdf}\xf7\xdcQ=\xf7\xb0\x00\x13\xdf{\x00\x0e<\x13\xb0@\x03a:\xf7\xcfE'
which ... doesn't look like it'd be a valid vCard line. It's not valid UTF-8 either. So I'm not sure what it's trying to do.

Could you please copy the whole problematic vCard into a comment? Maybe it's a continuation line or something ... With the complete input, hopefully we can figure this out.

VERSION:2.1
N;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:=E8=94=A1;=E5=AA=9B;;;
FN;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:=E8=94=A1=E5=AA=9B
LATESTDATE:null
COUNTRYISO:null
OPPO_RECENT_CALL:NULL
TEL;CELL:+8613810109388
EMAIL;WORK:[email protected]
ORG;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:=E5=8C=97=E4=BA=AC=E4=B8=AD=E8=88=AA=E6=99=BA=E7=A7=91=E6=8A=80=E6=9C=
=89=E9=99=90=E5=85=AC=E5=8F=B8
TITLE;CHARSET=UTF-8;ENCODING=QUOTED-PRINTABLE:=E4=BA=BA=E5=8A=9B=E8=B5=84=E6=BA=90=E9=83=A8=20=E4=BA=BA=E5=8A=9B=E8=
=B5=84=E6=BA=90=E9=83=A8=E7=BB=8F=E7=90=86
URL:www.zhztech.com
PHOTO;ENCODING=BASE64;JPEG:/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAIBAQEBAQIBAQE
 CAgICAgQDAgICAgUEBAMEBgUGBgYFBgYGBwkIBgcJBwYGCAsICQoKCgoKBggLDAsKDAkKCgr/
 2wBDAQICAgICAgUDAwUKBwYHCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKC
 goKCgoKCgoKCgoKCgr/wAARCABgAEgDASIAAhEBAxEB/8QAHAAAAgMBAQEBAAAAAAAAAAAABQ
 YABAcDAgEI/8QAQBAAAgEDBAAEAwMIBwkBAAAAAgMEBQYSAAcTIgEUMkIII1IRFTMhJGJygqL
 C0kNTc4SSk/AWJSYxNDVEg7Li/8QAGwEAAgIDAQAAAAAAAAAAAAAAAwYEBQACBwH/xAAzEQAC
 AQMBAwkGBwAAAAAAAAAAAgEDBBIFFTRxBhETFCIjMYGxJCUyMzVBUVJhYnKy8P/aAAwDAQACE
 QMRAD8A/Yu1Fk0uZLqVcqKfM+BNXGjx5Q5jHJfIREvL0kXNiX9mOnriAceg9dJOwdR85GrkIM
 fzOvcHUcf/AA45fxadJcpKcOQ/yl1D9LS3o1NVsUx/KFv5yuZOgx/HxLx8epa6RUL5M/APyjj
 obR79tSr12batMu2DIqtLIRqNPS8SbFIlrYPIPtyWxZdvaQ6JTKpHoECTWZqXEmKgmtGLGY9p
 CI5EK1rEmMLEeoiJERekctWuEEYO0mKlZB0xy66uV78tPBIY/aTfdpKpPxC7bLiVOdKukoYUP
 7yKf94QpEYiXBxGW5IsWJSUrJixJycl5MERIiIR0dou4VCvmmyW0A5h+FNrLqbK81TJEYfMJ6
 s4+ZY8i+34y8ll7S663wAuMbRBdnHh7sfT/baI7bIAqO5AK/Lyl2/ZHQmU/KzfH7MS8Rxz/wA
 zRDaZ5sgsA2F4kLy9WoErjeRwJlPdJMlvqh88OFI8OwEUgDWvLLrw/T6h+Z+7qav3jLkuoUGN
 HZj4plSCazD6lp6/UQ9fbqaTNfp+9anl6QXtg3sqmafDDUuefeoGeYrvIREvb/2umlph3TsZN
 +1624NbtmROpVPqw1RrItQ4hXKj/Mjcyv6dIsHk6lkLkx/lsHkJY7ZOgx6HBqcmNllOqzJDSL
 +kIVpXl6fpVp6jh4tDPj/a04aWuNnS4FFdtlcvIq2NYMKDuA2/65tQyPctUgEqfV4tfZUIMVa
 WcaEpGSSyWxi2chcKBHIXZMIsSY33ao2W9LcFOqkw0qF6qfR5nBJlMWQsFa2cixHIhEezBXj6
 ixy12V4GY+Pj9v2Y/vdteKtQqTckJ9ArlKjzqfUIrI82HMjiapCS6sWxZdSEhIhIS+rVsBEe+
 LxoVnn9+07aPdSuxilMnSpVDmzi45EVxEKfLMlLYS2MbjxrSSWDjkJJXksFO3Psv72uCjVTZH
 faNGTWWU2bUCrtYQpg+aH84hEM8fkkSyZkntwrZ1xIVs0uh7E7UUOAFLo+1dDpiFkvGPS6auI
 I4kwsfkiPX5zhIfSQuYJCQsISvb0eKU2ytXp45i8cevtYX8OvafabEwMWNY9k1+BGrVIqNeNa
 8h8vKuWcwcuQmFyLJxCRZMLHLLriI9RERercoUahrIIzyLwLsXJj6tYbtjezrMqK5GEhqHKxl
 L+of6zIvpLLW70OfGnxlS4x5KYoSUX1CXu15Wt+jfI0yZUxMm3IsiqW34ITVIqeP5hKcto4u/
 DyHH1fTqam4N81GuwYSau5bSWTCUWIrIvT+qPUhH/Fqa5/rmW1Knl6QM+n7qokbSBHO3vsT/y
 KVI/axcQ6tUXcuDOv6ZYdOrFNqDafC56smHKEpNLYXCSFvWJFjzCxjF5Y9Uljl7R2yLwl24Lu
 TLw8/OH6vTKcP8OmCDuHZ8i5jsaNedLOtqh+ebRlzVlLGOLOPm4ssuMWdcsccuum2w3Wlwj0F
 +vvD/ykpWruDuNW5rIlQ2GqFFQUBz4sqrViGXzBJYrjuGMxxLIiJhZDyCK1iWWTOMStjXTetb
 o0Sr3Btp9zymNkDIpsqqraxIi4hUWSBIS5BEWer5fJj2LVWDvjtVKixJYbtW6aJ0qPDhuGrpw
 kSJCRkISssuzGJIWCPqISEh6lqxbu+O1FySjp9vbxW3NlLbg2PBrUdrFs5GDiQiWQlkhv+Sz6
 S1MTwIzeB2q167gU6qU+LSNm5E+HIrKY8+aurxVFDhkMjKZxkXzMSWn5Y9iGQJD2El6vbzJkD
 byy8Ay8PPjh+txs7a7U7cyw3UyPcSdwKP40yRFKRHnjUFkhyRTzE4WZYkvj+Zl6ce2udR3C2b
 vCvusB251BdW4I88yjx60vzcUew8jEiWQjkWORDjkX6WpFNsXyPTra9k/7V2DClpATnx+QVSP
 TyDzF1L/Xq/WLRHZm+XW+7xt2uGwUk3FXIHZLPpx9v8JfraYLFosKkUqPToDycoe6pBEJZZER
 ZZD192h25tgeMj/immJLxPD88Ee2I/1g/T+z/No3SK8yrBBHugl/d8cPBBF3YeI9O2PpyLt/9
 fu6mpfDTTTExQPowizZj2EhHrjqa5vr/wBXq+X9YGTT1nqiiN8LbPF1gD459l16tK6/o1SYP8
 Ove2VFpjt4Llrvh8LMW36gJlDC+xjwxfWU4x2YkwfnYkRdfxB/NyyJZCK9T4YYjYW3wKd4fab
 KzWHnj7uSpSWfxaebXuWi3DynQq1FmcMpyGlHYJCLEs42D+sLBJZfSQkPqEtOFiuNskftgX7h
 sq7T+omUPZHaKrTfCj1T4MaTGi0l6Tp0qZQ6SSCIkrEmJEWExfGKEJLIR9KxHIRyFgq+0W3sK
 dT/AAh/DzSZ5sa6D5iHSoP+748gS52FyEv5LPSwV5EXJ6SHLG1de9+1FjypNIu7dq26PJhwxl
 TY9SrUdDI8cmLWLmCwshWTGLXl6cmCPuHRKRu5tzToDq7UNy6DHhLhJmtnSKqlakw2ZEtxMIs
 eNgrYQl6S4y+nViRsYAk60rT2+o0yHavwfFU44zSgnSbfp9HR56PIjpJrBGTISvhLjWlgsISI
 o4/LJYiWiW4OwNlx2Nvmy9oaGm55UjinVeDSkrmOjl2YJOERYQ5Csse3Ycvbpmp182vLjlLi3
 hTSR95FD5lzV4lIGR5ck5ZfiDIEkkPq5Ovq666W3vJtjX1xl0bdW35xVCKyVB8rWEn5hIivJi
 8S7LEXJ7D1+cv6h1rTbFueApndCmV6hVHxBM2VBMcRavPAy7e4f9fva0G1d5bijcZ1QFzsfxS
 wwIsvpIevq/R1f3chpk2yGZLHGaJiOGX1DqhtjYES7qZJkTKgxMpbcOQey8cfpLU3Kk9PnZQY
 C36rltVCDElUugFDZz4zcj4+vX8P/D9Pt9Opq3vhtvOtyjpjBL8wEyRxKGOBZY+rIh9vbHHtq
 a5br2O1anl6QNOnSvU0/wB9xI2jimu1YOeXzIonkXq7dv4tL/wwWr8OdCv6+5mzSIoXFUKsL7
 48vVWSSKV5icXZZOYKfnFJLERH1Y45DiLLtI/zFj0NweGXgyiQzyyy9SR92mmg1qkVR8yFT6x
 Hksp7xjTlpkCRR3cYs42CPpLjYtmJe1gl7h092vZoLwFl/msZLdd37Xzot4Kq3wLXNWkQ6XS5
 VZErGisG4iYKyTHQLCHzrkizsOOKeMhyyxErNxVjZGn2RUKlJ+ByvVOGm3IBtocPbdbXylsGG
 I08UEPZkcVwMll8seEREiKK4U7WKx8O59sR76vx4+PyQ69/bqYDbwMu2/RsVc1Yum2oXwqSqd
 LqlXci7PvTbgoyK4S5zE+aZJJPDPWwmE4SFjC42ERCJZDpkvDY20rTtCJB2422gwAiyPzeDQ6
 etQpWQjlxrWIiI/LSP/rH6R09xVJHxIPDVO/rll2dRUVODFSRk8QPmAsccSIvd+jrKeSt2Qpm
 lLnTlRzhDOYKeUvFsXlIci9pF7ctOu2d++NoG6MEJbkuMSx5cCHHrphsusWvuZCd5+hr5l9Wr
 Z2MR9uJY/y/zW52x9Fm+H2UuouiZFliXzB/m/e1JarSaMXUGC95Ljpd52eFQp8d2cGqLElkfG
 QiS2F6hy6lxjqaFblWPWrZtk4NSSmQiVNSXIlv9IsWY+r9r9HU1zDlJTp7UbhAx6fuimafD3V
 Am7U2tM5R8ea2oJ5f3deufw7WXtJQK1dU6wNrnUOZOrLEVqoOVkutELnM5luFjBYIsa9ePVi+
 PjJaxWsdCPhHn82ydmeHif25WbSSy/uq9aFUpk+lUaZNolDKdMGKxqIKSWJSnCPVeREI5FiI9
 iEfqIdP9uvdKL7/ABSZ3vLXPhXRfcejbmHXIc+i1yl1OKRU+sIpsWZlMcqVzLWMXtzSycwixL
 jLm/B+XzsC1fhLue+pDbXvaZBqMOhjJkUacDoPkUpqgzJMhgy0i5bGSBRzEwsiEUl16kR27dq
 ZUEG3XB+GixbirX3lNb/0iUPZF4ZzkDzsEsXMlMESL0/nTmfVk8bcLk3QK77ujZ1dv17y7IYD
 MOO+cmLyCWJOSTBxIhFnGLCH05duoycDQB07Z7Zu6b3jXlS9xHPuG1bjrT5EiHV1sYt0qKK5M
 WSOJZCtbIbBFnZfHHxLHqSdvJ8Puxs2mU+iS99qpUuO96PX7eo1Qupbxpk6OtxL8osuwpcMdw
 lHEiSIrZwrTj1dtuHVkKBHRcPwTjS2XdXqg256fS51Jkri8jOEpk8iYnmJycSLhGQWI4llpt3
 5se2qzb8R1VteDIZFq6ZEVjoi2Ely1uFbFkXpIRYwRIfTkX1a3pr3qmCJtpu/btLmwanRr9ps
 vz3H5JMWoLLzwsSUgRHEiyySsmD9QjyD17a3K1N37AuRseHTL+ppy5jyQqD51fOThEiJPHlly
 CK2EQ/Ssi9I6Sdv/h/2/rG38JNMsKiwZMfmKHKj0xayTkxeQ9R9JcEf/JT/AFY647Vqp219wS
 yOgR4fjUJRFVijxxFjHCIr5iIfUWIiOX0iP0jo1VOmWfxUw+bhVypcMmluqjjQuUQqSTS9uQi
 Il7fVj+1qarX2K5smWaT5fDzTCBg4l1y9v/51Nc55SQu054QX1i09Wg//2Q==

END:VCARD

Apr 25 '24 14:04 yangtfu

@yangtfu, I've run into a couple of issues testing that card.

First, it's missing the BEGIN:VCARD at the start. Perhaps that's just a cut-and-paste error? If not, then the encoder is pretty broken, and I'm not sure what could be done about parsing it.

Next, if I add a BEGIN:VCARD to the start, it is parsed successfully using Python 3.7 and current vObject.

Last, the problematic string =30=34=33=33=3F=E9=97=AA=E9=97=AA=48=E7=BA=A2=E6=98=9F from the previous report doesn't seem to be in this card at all. Can you please check that this is the card that's causing the exception?

Apr 25 '24 23:04 da4089

@da4089 It's a paste error. The file is quite long. I'll put it here

Apr 26 '24 01:04 yangtfu

Thanks @yangtfu, that's very helpful.

I am able to reproduce the problem. @pbiering, the problem is here:

  File "/home/ytf/ws/py_ws/Radicale/radicale/item/__init__.py", line 52, in read_components
    return list(vobject.readComponents(s))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If that's changed to return list(vobject.readComponents(s, allowQP=True) the sample file is processed correctly in my test case code. I'm not across the history of why allowing quoted-printable encoding is disabled by default in vobject, but I think this will likely be a frequent source of problems for those whose cards use non-ASCII characters encoded for vCard 2.1 (which required 7-bit clean data).

I'm also unaware of any past issues this might have had in Radicale: perhaps it's disabled for a reason here?

Either way, I've sent a PR (#1476) with the above change. It's had no testing in Radicale.

Apr 27 '24 14:04 da4089

Thank you for tracing down the issue.

I'm also unaware of any past issues this might have had in Radicale: perhaps it's disabled for a reason here?

Potentially nobody so far was aware about.

Either way, I've sent a PR (#1476) with the above change. It's had no testing in Radicale.

Can you create a 2nd PR with extending the test cases?

Apr 27 '24 14:04 pbiering

PR merged, please test.

Once test case is added, this issue can be closed.

Apr 27 '24 15:04 pbiering

Radicale Radicale copied to clipboard

radicale upload VCARD2.1 failed

Radicale
Radicale copied to clipboard