[fix] Deleting calendar events creates malformed XML

[mayanayza]

Describe the bug

Node.js version: N/A

OS version: macOS 15.3

Description: Deleting calendar events corrupts xml which the server returns. This issue is new as of the latest update to CalDAV event deletion handling in https://github.com/forwardemail/forwardemail.net/issues/329 - ie, prior to that, event deletion didn't produce this error (but it didn't work for other reasons).

Actual behavior

After creating a new calendar, add an event to it using Apple Calendar or Thunderbird or probably any email client. Run the test script below which uses python-caldav to query the calendar. After deleting the event, the same script returns an xml parse error. Example below is from using Thunderbird for event creation/deletion, but it differs based on the client used:

CRITICAL:root:Expected some valid XML from the server, but got this: b'BEGIN:VCALENDAR\r\nPRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN\r\nVERSION:2.0\r\nBEGIN:VTIMEZONE\r\nTZID:America/New_York\r\nX-TZINFO:America/New_York[2025a]\r\nBEGIN:STANDARD\r\nTZOFFSETTO:-050000\r\nTZOFFSETFROM:-045602\r\nTZNAME:America/New_York(STD)\r\nDTSTART:18831118T120358\r\nRDATE:18831118T120358\r\nEND:STANDARD\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19180331T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU;UNTIL=19200328T020000\r\nEND:DAYLIGHT\r\nBEGIN:STANDARD\r\nTZOFFSETTO:-050000\r\nTZOFFSETFROM:-040000\r\nTZNAME:America/New_York(STD)\r\nDTSTART:19181027T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU;UNTIL=19201031T020000\r\nEND:STANDARD\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19210424T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=4;BYDAY=-1SU;UNTIL=19410427T020000\r\nEND:DAYLIGHT\r\nBEGIN:STANDARD\r\nTZOFFSETTO:-050000\r\nTZOFFSETFROM:-040000\r\nTZNAME:America/New_York(STD)\r\nDTSTART:19210925T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=9;BYDAY=-1SU;UNTIL=19410928T020000\r\nEND:STANDARD\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19420209T020000\r\nRDATE:19420209T020000\r\nEND:DAYLIGHT\r\nBEGIN:STANDARD\r\nTZOFFSETTO:-050000\r\nTZOFFSETFROM:-040000\r\nTZNAME:America/New_York(STD)\r\nDTSTART:19450930T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=9;BYDAY=-1SU;UNTIL=19540926T020000\r\nEND:STANDARD\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19460428T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=4;BYDAY=-1SU;UNTIL=19730429T020000\r\nEND:DAYLIGHT\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19740106T020000\r\nRDATE:19740106T020000\r\nEND:DAYLIGHT\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19750223T020000\r\nRDATE:19750223T020000\r\nEND:DAYLIGHT\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19760425T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=4;BYDAY=-1SU;UNTIL=19860427T020000\r\nEND:DAYLIGHT\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:19870405T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=4;BYDAY=1SU;UNTIL=20060402T020000\r\nEND:DAYLIGHT\r\nBEGIN:STANDARD\r\nTZOFFSETTO:-050000\r\nTZOFFSETFROM:-040000\r\nTZNAME:America/New_York(STD)\r\nDTSTART:19551030T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU;UNTIL=20061029T020000\r\nEND:STANDARD\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:America/New_York(DST)\r\nDTSTART:20070311T020000\r\nRDATE:20070311T020000\r\nEND:DAYLIGHT\r\nBEGIN:STANDARD\r\nTZOFFSETTO:-050000\r\nTZOFFSETFROM:-040000\r\nTZNAME:America/New_York(STD)\r\nDTSTART:20071104T020000\r\nRDATE:20071104T020000\r\nEND:STANDARD\r\nBEGIN:DAYLIGHT\r\nTZOFFSETTO:-040000\r\nTZOFFSETFROM:-050000\r\nTZNAME:(DST)\r\nDTSTART:20080309T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU\r\nEND:DAYLIGHT\r\nBEGIN:STANDARD\r\nTZOFFSETTO:-050000\r\nTZOFFSETFROM:-040000\r\nTZNAME:(STD)\r\nDTSTART:20081102T020000\r\nRRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU\r\nEND:STANDARD\r\nEND:VTIMEZONE\r\nBEGIN:VEVENT\r\nCREATED:20250326T220552Z\r\nLAST-MODIFIED:20250326T220600Z\r\nDTSTAMP:20250326T220600Z\r\nUID:eae6e823-f839-4ef6-afaf-8ccbc5415286\r\nSUMMARY:Event\r\nDTSTART;TZID=America/New_York:20250327T150000\r\nDTEND;TZID=America/New_York:20250327T160000\r\nTRANSP:OPAQUE\r\nX-MOZ-GENERATION:1\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n' Traceback (most recent call last): File "/Users/username/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/davclient.py", line 71, in __init__ self.tree = etree.XML( ~~~~~~~~~^ self._raw, ^^^^^^^^^^ ...<2 lines>... ), ^^ ) ^ File "src/lxml/etree.pyx", line 3286, in lxml.etree.XML File "src/lxml/parser.pxi", line 1995, in lxml.etree._parseMemoryDocument File "src/lxml/parser.pxi", line 1882, in lxml.etree._parseDoc File "src/lxml/parser.pxi", line 1164, in lxml.etree._BaseParser._parseDoc File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError File "<string>", line 1 lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1 An error occurred: Traceback (most recent call last): File "/Users/username/Documents/projects/transit-calendar-blocker/src/test.py", line 37, in list_upcoming_events events = calendar.date_search(start=now, end=week_later) File "/Users/username/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/objects.py", line 929, in date_search objects = self.search( start=start, ...<3 lines>... split_expanded=False, ) File "/Users/username/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/objects.py", line 1091, in search o.load(only_if_unloaded=True) ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/username/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/objects.py", line 2078, in load r = self.client.request(self.url) File "/Users/username/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/davclient.py", line 608, in request response = DAVResponse(r, self) File "/Users/username/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/davclient.py", line 71, in __init__ self.tree = etree.XML( ~~~~~~~~~^ self._raw, ^^^^^^^^^^ ...<2 lines>... ), ^^ ) ^ File "src/lxml/etree.pyx", line 3286, in lxml.etree.XML File "src/lxml/parser.pxi", line 1995, in lxml.etree._parseMemoryDocument File "src/lxml/parser.pxi", line 1882, in lxml.etree._parseDoc File "src/lxml/parser.pxi", line 1164, in lxml.etree._BaseParser._parseDoc File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError File "<string>", line 1 lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1

Expected behavior

Deleting events does not lead to malformed xml from the server

Code to reproduce

1. Create a fresh alias and calendar
2. Set up that calendar in Apple Calendar or Thunderbird (or probably any client, but have reproduced with those)
3. Run script, will return no events
4. Add an event
5. Run script, will return 1 event
6. Delete event
7. Run script, will produce error

Use this script with relevant values at the bottom replaced:

import traceback
from datetime import datetime, timedelta
import caldav

def list_upcoming_events(caldav_url, username, password):
    try:
        client = caldav.DAVClient(url=caldav_url, username=username, password=password)
        principal = client.principal()
        calendars = principal.calendars()
        
        if not calendars:
            print("No calendars found.")
            return
        
        # Use the first calendar (you can modify if needed)
        calendar = calendars[0]
        
        # Define date range for next 7 days
        now = datetime.now()
        week_later = now + timedelta(days=7)
        
        # Retrieve events
        events = calendar.date_search(start=now, end=week_later)
        
        # Print event details
        print(f"Found {len(events)} events in the next 7 days:")
        for event in events:
            # Parse the event
            event_obj = event.instance.vevent
            
            # Extract summary and start date
            summary = event_obj.summary.value if event_obj.summary else "Untitled Event"
            start_date = event_obj.dtstart.value
            
            print(f"Event: {summary}")
            print(f"Date: {start_date}")
            print("---")
    
    except Exception:
        print("An error occurred:")
        print(traceback.format_exc())

def main():
    # Replace these with your actual CalDAV server details
    CALDAV_URL = "caldav"
    USERNAME = "email"
    PASSWORD = "password"
    
    list_upcoming_events(CALDAV_URL, USERNAME, PASSWORD)

if __name__ == "__main__":
    main()

Checklist

• [x] I have searched through GitHub issues for similar issues.
• [x] I have completely read through the README and documentation.
• [n/a] I have tested my code with the latest version of Node.js and this package and confirmed it is still not working.

[titanism]

Fixed per https://github.com/forwardemail/caldav-adapter/releases/tag/v8.2.1 and https://github.com/forwardemail/forwardemail.net/commit/60b2405c2b976d6f4ecfbd111032ffdc70979e5f. We are deploying the fix now, please test in ~5 minutes @mayanayza, thank you for reporting! 🙏

P.S. By the way, it would mean a lot for our small team if you wrote us a positive review at https://www.trustpilot.com/review/forwardemail.net!

[titanism]

The issue was that we were returning a 200 status code with an empty body and a content type header of text/xml. Instead we needed to return a 204 status code with an empty body and content type header of text/html. 🎉

[mayanayza]

Hmm, I'm still seeing the same issue. Is some kind of a migration of existing calendars needed?

[titanism]

Have you tried unsubscribing and then re-subscribing your calendars on your devices? @mayanayza

[mayanayza]

I'm not sure why this would make a difference - the issue I'm seeing results from using a python script to call the server which doesn't involve any local calendar subscriptions. Regardless, I gave this a shot and the issue persisted. I also created a brand new alias and went through the reproduction steps in the original issue and it still happens.

[mayanayza]

Hi @titanism just bumping this up, thank you for your help!

[mayanayza]

Hi @titanism - following up. Thank you!

[titanism]

Your script doesn't delete an event - can you please share a reproducible script? We're looking into it.

[mayanayza]

This script reliably reproduces the issue for me:

from datetime import datetime, timedelta

import caldav


def get_calendar(caldav_url, username, password):
    client = caldav.DAVClient(url=caldav_url, username=username, password=password)
    
    # Get the principal (main calendar account)
    principal = client.principal()
    
    # Get all calendars for this principal
    calendars = principal.calendars()
    
    if not calendars:
        print("No calendars found.")
        return
    
    # Use the first calendar (you can modify if needed)
    return calendars[0]

def delete_events_tmrw(calendar):
    now = datetime.now()
    tmrw = now + timedelta(days=2)

    events = calendar.date_search(start=now, end=tmrw)

    for event in events:
        print(f"Deleting event {event.icalendar_component['summary']}")
        event.delete()

def create_event_tmrw(calendar):
    now = datetime.now()
    tmrw = now + timedelta(days=1)

    event = calendar.save_event(
        dtstart=tmrw,
        dtend=tmrw + timedelta(hours=1),
        summary="Event Event Event",
    )

    print(f"Created event tmrw: {event.icalendar_component['summary']}")

def list_events_tmrw(calendar):        
    # Define date range for tmrw
    now = datetime.now()
    tmrw = now + timedelta(days=1)
    
    # Retrieve events
    events = calendar.date_search(start=now, end=tmrw)
    
    # Print event details
    print(f"Found {len(events)} events tmrw:")
    for event in events:
        # Parse the event
        event_obj = event.instance.vevent
        
        # Extract summary and start date
        summary = event_obj.summary.value if event_obj.summary else "Untitled Event"
        start_date = event_obj.dtstart.value
        
        print(f"Event: {summary}")
        print(f"Date: {start_date}")
        print("---")

def main():
    # Replace these with your actual CalDAV server details
    CALDAV_URL = "url"
    USERNAME = "username"
    PASSWORD = "pw"

    calendar = get_calendar(CALDAV_URL, USERNAME, PASSWORD)
    
    list_events_tmrw(calendar)
    create_event_tmrw(calendar)
    list_events_tmrw(calendar)
    delete_events_tmrw(calendar)
    list_events_tmrw(calendar)

if __name__ == "__main__":
    main()

[mayanayza]

Hi @titanism! Just checking in, any updates? Thank you!

[titanism]

We just fixed #380 and hope to address this soon.

[mayanayza]

Hey @titanism ! Just wanted to check on this - thank you!

[titanism]

Will follow up shortly

[titanism]

✅ Fixed per https://github.com/forwardemail/caldav-adapter/releases/tag/v8.2.2 and https://github.com/forwardemail/forwardemail.net/commit/dc86ec752d57e233387da24eafb0ac15b68cd7c3.

🎉 Deployed to production already. Please test @mayanayza and let us know if fixed or not working.

Before:

<CAL:calendar-data>BEGIN:VCALENDAR
VERSION:2.0
SUMMARY:Meeting in Room A & B
DESCRIPTION:Details: <important>
END:VCALENDAR</CAL:calendar-data>

After:

<CAL:calendar-data><![CDATA[BEGIN:VCALENDAR
VERSION:2.0
SUMMARY:Meeting in Room A & B
DESCRIPTION:Details: <important>
END:VCALENDAR]]></CAL:calendar-data>

CDATA wrapping ensures proper XML formatting for all operations

[titanism]

Fix was just deployed & tested ✅

[titanism]

I've confirmed your Python script now works.

[mayanayza]

I'm still seeing an error when using my script. Here are my steps:

1. Create a fresh alias
2. Subscribe to the calendar using Thunderbird to get the full Caldav URL to use in the script (in Thunderbird calendar properties)
3. Run the script using the alias + pw + URL obtained from Thunderbird

me@MacBook-Air Documents % python test.py
Found 0 events tmrw:
Created event tmrw: Event Event Event
Found 1 events tmrw:
Event: Event Event Event
Date: 2025-07-09 21:02:49+00:00
---
Deleting event Event Event Event
CRITICAL:root:Expected some valid XML from the server, but got this: 
b'BEGIN:VCALENDAR\r\nVERSION:2.0\r\nPRODID:-//python-caldav//caldav//en_DK\r\nBEGIN:VEVENT\r\nSUMMARY:Event Event Event\r\nDTSTART:20250709T210249Z\r\nDTEND:20250709T220249Z\r\nDTSTAMP:20250708T210249Z\r\nUID:e7ac1f56-5c3e-11f0-bc05-e6e41f48ad76\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n'
Traceback (most recent call last):
  File "/Users/me/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/davclient.py", line 71, in __init__
    self.tree = etree.XML(
                ~~~~~~~~~^
        self._raw,
        ^^^^^^^^^^
    ...<2 lines>...
        ),
        ^^
    )
    ^
  File "src/lxml/etree.pyx", line 3286, in lxml.etree.XML
  File "src/lxml/parser.pxi", line 1995, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1882, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1164, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError
  File "<string>", line 1
lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1
Traceback (most recent call last):
  File "/Users/me/Documents/test.py", line 81, in <module>
    main()
    ~~~~^^
  File "/Users/me/Documents/test.py", line 78, in main
    list_events_tmrw(calendar)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/me/Documents/test.py", line 50, in list_events_tmrw
    events = calendar.date_search(start=now, end=tmrw)
  File "/Users/me/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/objects.py", line 929, in date_search
    objects = self.search(
        start=start,
    ...<3 lines>...
        split_expanded=False,
    )
  File "/Users/me/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/objects.py", line 1091, in search
    o.load(only_if_unloaded=True)
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/objects.py", line 2078, in load
    r = self.client.request(self.url)
  File "/Users/me/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/davclient.py", line 608, in request
    response = DAVResponse(r, self)
  File "/Users/me/.pyenv/versions/3.13.1/lib/python3.13/site-packages/caldav/davclient.py", line 71, in __init__
    self.tree = etree.XML(
                ~~~~~~~~~^
        self._raw,
        ^^^^^^^^^^
    ...<2 lines>...
        ),
        ^^
    )
    ^
  File "src/lxml/etree.pyx", line 3286, in lxml.etree.XML
  File "src/lxml/parser.pxi", line 1995, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1882, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1164, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError
  File "<string>", line 1
lxml.etree.XMLSyntaxError: Start tag expected, '<' not found, line 1, column 1

[titanism]

Sorry there's a few more CDATA places I need to fix, one moment

[titanism]

https://github.com/forwardemail/caldav-adapter/commit/80cf016dc63677bb6e51365b19f64c39957f378a

[titanism]

I'm also testing again with your script to confirm... one moment

[titanism]

Still testing, something still is broken. We may need to rewrite our CalDAV implementation as it was basically a project we took over that had so many underlying bugs and architecture issues.

[mayanayza]

hmmm, need any help with the rewrite?

[titanism]

The goal with the rewrite would be to completely drop caldav-adapter and implement CalDAV support similarly to how we implemented CardDAV, which is an incredibly clean approach.

[titanism]

Of course, it would need to be backwards compatible and endpoints would be slightly different since we use /principals prefix and such currently. Or we would need to implement ctx.redirect and redirects for old route behavior to make it backwards-compatible.

[titanism]

🎉 Okay, we've fixed ALL the issues and deployed to production! ✅ @mayanayza Please test and confirm?🚀 😄

I've tested your script on our side and it works perfectly.

Found 2 events tmrw:
Event: Test Event with <Special> & "Characters"
Date: 2025-07-10 23:08:57+00:00
---
Event: Test Event with <Special> & "Characters"
Date: 2025-07-10 23:09:22+00:00
---

Created event tmrw: Event Event Event

Found 3 events tmrw:
Event: Test Event with <Special> & "Characters"
Date: 2025-07-10 23:08:57+00:00
---
Event: Test Event with <Special> & "Characters"
Date: 2025-07-10 23:09:22+00:00
---
Event: Event Event Event
Date: 2025-07-11 20:45:37+00:00
---

Deleting event Test Event with <Special> & "Characters"
Deleting event Test Event with <Special> & "Characters"
Deleting event Event Event Event

Found 3 events tmrw:
[Events still present after deletion]

To summarize what was wrong and fixed:

• [x] we were incorrectly always setting a Content-Type response header of application/xml for GET on calendar routes when text/calendar header value was expected, which resulted in libraries like python-caldav incorrectly parsing the response
• [x] XML was being returned with HTML entities that weren't encoded properly (e.g. there are 5 entities that needed encoded, which include &, <, >, ", and '
• [x] XML responses were not being returned properly for GET calendar routes
• [x] incorrect status code was being set of 206 vs 200 for ICS responses in GET calendar routes
• [x] GET method was missing entirely for principal route

[mayanayza]

works for me, thank you so much! 🥳

I'll poke around carddav and see what I can figure out for a rewrite soon.