cal.com
cal.com copied to clipboard
calcom
[CAL-928] RFC: User availability caching
About caching availability. Already discussed with @alannnc and @emrysal and so far the best approach to us would be:
- Create a custom Cal.com CalDav server
- Each Cal.com user will have an account there by default
- The CalDav account cannot be removed by the user (it won’t be shown as installed in apps) as is internal only.
- we can sync all calendars integrations into the user CalDav via webhook and crons to keep it fresh
- when checking availability, we only check against our CalDav which should have the user availability already synced and combined. (We also have control into the CalDav server instance since it’s ours)
- response times should be instant and should not scale as we add more calendars since we’re going to check only 1 calendar (ours) at booking time
PROS
- Faster availability checks
- Easier caching
- We can user this calendar to show an actual calendar view in our dashboard. I think Sean already coded the UI.
- You can subscribe to this calendar via URL and get an actual combination of all your calendars in a single url. IE you can see this calendar on your iCal, gCal, etc.
CONS
- It’s and XXL task
- Availability could not be 100% fresh
- Harder adoption for self hosters? (It will depend on implementation)
- Should be opt-in? (Feature flags will help with this) So we can give a choice between the new or old algorithm
Additional details
- We could auto assign a permanent
[email protected]calendar address - Address could be aliased with username:
[email protected] - the benefits of solving the cached availability with caldav is that people technically dont need to connect a single calendar to get started
- We could use our own calendar as a "default caldav" calendar yes, but at the moment it works without a dedicated calendar connected (we have a lot of logic in availability/booking that handles this)
- You could actually use the calendar address for invites. i.e. you use someone else scheduling link and you put your calendar email in there
- Shipping a native CalDav can fix a lot of issues + we can sell it as a feature ("get a scheduling product + a calendar for free") and people can self-host cal with a functioning caldav
- You could actually subscribe your Cal availability to other calendars via URL like:
webcal://[email protected]/published/TOKEN
From SyncLinear.com | CAL-928
how can we ensure to keep availability fresh and make sure people dont get double booked?
If the webhooks/callbacks are all on point, whenever an event is added/removed from any of the connected calendar apps, it should automatically update such events on the CalDAV calendar, thus making it fresh/reliable
will send this to a couple external contributors to get their thoughts on this first before we start implementing
As a Cal user, I'd say, for me, the 'availability being 100% fresh' is quite important.
I like the idea of having an 'aggregated' calendar available.
As a self-hoster, I'd say that, while this does increase deployment complexity, this is exactly the kind of thing Docker Compose is designed to handle well. Take, for example, Baserow which implements multiple, distinct services (a web frontend service, a backend service, a few different database services, some worker services, and a supporting Caddy instance). https://gitlab.com/bramw/baserow/-/blob/develop/docker-compose.yml
What @PeerRich mentioned vibes with me as the right approach, we add a public "CalendarEvent" table which functions as a caching layer, normalized so to prevent any joins for optimal performance. In this we add freshness information, and aim to keep it at least up to date to 5 minutes.
No - CalDav is the access protocol, it doesn't say anything on how the server stores or caches the calendar.
From my limited knowledge here and just based on what I read above, the approach seems great. The biggest downside is the potential of being double booked due to the caldav server going out of sync with other calendars.
I believe we can solve that by not simply relying on webhooks, but to have a background service that syncs the data with each of the connected calendars in a polling interval. We should make that configurable with sensible defaults like fetching every 5min (assuming they connected the calendars with API keys or some way to fetch it, as I'm not yet up to speed on the approach being used here).
Just my 2c. If we plan on only using webhooks and relying on them, we need to make damn sure we don't end up supporting some 3rd party calendar that is unreliable and then the end-user assumes cal.com is slow/unreliable.
From my limited knowledge here and just based on what I read above, the approach seems great. The biggest downside is the potential of being double booked due to the caldav server going out of sync with other calendars.
I believe we can solve that by not simply relying on webhooks, but to have a background service that syncs the data with each of the connected calendars in a polling interval. We should make that configurable with sensible defaults like fetching every 5min (assuming they connected the calendars with API keys or some way to fetch it, as I'm not yet up to speed on the approach being used here).
Just my 2c. If we plan on only using webhooks and relying on them, we need to make damn sure we don't end up supporting some 3rd party calendar that is unreliable and then the end-user assumes cal.com is slow/unreliable.
That's an interesting suggestion, thanks 🙏
I've had some experience with polling in various fields of work, from time management, to regular scheduling and what I've found unequivocally is, polling is almost never the best idea as it always leaves the idea of potential double booking if new bookings are made in that polling interval (5 mins being too much for many users, who are booked quite frequently). Webhooks (when reliable) are definitely the best way forward to ensure consistent data synchronization, in my opinion.
i wonder if we can increase webhook deliverability with something like https://www.svix.com/ or https://getconvoy.io/ if we choose to go down the webhook approach
i wonder if we can increase webhook deliverability with something like https://www.svix.com/ or https://getconvoy.io/ if we choose to go down the webhook approach
The only things they add on top of what we already have (from what I can see) are:
- Retries on failed requests (We've had plans on adding that for a while, so might be worth doing that)
- Idempotent retries (this is quite neat, but if we plan the retry model well, that should take care of this)
- Saving payload for a brief period in time. This can be added to our system as well on a low priority basis as this doesn't really do much in terms of deliverability Based on this, I'd probably say they don't add as much value in the given context of deliverability. Although, this is just after taking a quick look at the two services.
that sounds like we can do inhouse no? or do we need one of those services
that sounds like we can do inhouse no? or do we need one of those services
Yeah, I believe so. That was the plan anyway.