MagicMirror icon indicating copy to clipboard operation
MagicMirror copied to clipboard

[Bug] CALENDAR_EVENTS are being resend all the time with no updates

Open michaelarnauts opened this issue 1 month ago • 17 comments

Environment

MagicMirror² version: v2.33.0 Node version: v22.21.1 npm version: 10.9.4 Platform: Docker

Which start option are you using?

node --run server

Are you using PM2?

I don't know

Module

calendar

Have you tried disabling other modules?

  • [ ] Yes
  • [x] No

Have you searched if someone else has already reported the issue on the forum or in the issues?

  • [x] Yes

What did you do?

Configuration
var config = {
    address: '0.0.0.0',
    port: 8080,
    ipWhitelist: [],
    language: 'nl',
    timeFormat: 24,
    units: 'metric',
    logLevel: ["LOG", "WARN", "ERROR"],
    modules: [
        {
            module: 'alert',
        },
        {
            module: 'clock',
            position: 'top_right',
            config: {
                displaySeconds: false,
                dateFormat: 'dddd LL'
            }
        },
        {
            module: 'calendar',
            position: 'bottom_left',
            config: {
                fetchInterval: 600000,
                maximumNumberOfDays: 2,
                maxTitleLength: 50,
                fade: false,
                getRelative: 0,
                urgency: 0,
                timeFormat: 'dateheaders',
                dateFormat: 'dddd D MMMM',
                fullDayEventDateFormat: 'ddd D MMMM',
                broadcastEvents: false,
                sliceMultiDayEvents: true,
                colored: true,
                defaultSymbolClassName: '',
                calendars: [
                    {
                        name: 'xxx',
                        url: 'https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics',
                    },
                    {
                        name: 'xxx',
                        url: 'https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics',
                    },
                    {
                        name: 'xxx',
                        url: 'https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics',
                    },
                    {
                        name: 'xxx',
                        url: 'https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics',
                    },
                    {
                        name: 'xxx',
                        url: 'https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics',
                    },
                    {
                        name: 'xxx',
                        url: 'https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics',
                    },
                    {
                        name: 'xxx',
                        url: 'https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics',
                    },
                    {
                        name: 'Feestdagen',
                        url: 'https://calendar.google.com/calendar/ical/nl.be%23holiday%40group.v.calendar.google.com/public/basic.ics',
                    }
                ]
            },
        }
    ]
};

Steps to reproduce the issue:

I run magic mirror as a screensaver with fully kiosk. Therefore, the page is reloaded numerous times during the day. I think it schedules for CALENDAR_EVENTS somehow. After a while (it can be a day or more), i can see it starts to update the calendar (you can see it fade out and fade back in) the whole time.

When looking at the websocket (in my browser, it gets the same websocket messages), i can see a lot of those messages being send:

42/calendar,["CALENDAR_EVENTS",{"id":"module_2_calendar","url":"https://calendar.google.com/calendar/ical/xxx/private-yyy/basic.ics","events":[{"title":"Turnen","startDate":"1763661600000","endDate":"1763665200000","fullDayEvent":false,"recurringEvent":true,"firstYear":2025,"location":"xxx","geo":false,"description":false},{"title":"Vioolles bij xxx","startDate":"1763746200000","endDate":"1763748000000","fullDayEvent":false,"recurringEvent":true,"firstYear":2025,"location":false,"geo":false,"description":false}]}]

I have 7 calendars and they get pushed all the time with the same values.

When i restart magic mirror, the calendar stops refreshing as it should be. I believe it starts when the calendar is being refreshed.

What did you expect to happen?

I would expect updates to be send at midnight, or when an event has finished.

What actually happened?

After a while, it starts to send the same update over and over.

Additional comments

No response

Participation

  • [ ] I am willing to submit a pull request for this change.

michaelarnauts avatar Nov 20 '25 19:11 michaelarnauts

we do not know what the prior events are . We send ALL events discovered in the fetch cycle. One calendar url at a time. They can have different refresh times too_

You can disable sending expired events with a config Parameter

After the event time it will move to the expired list but if the calendar source sends us the events we process and send them out

The communications between browser and back end is web sockets

sdetweil avatar Nov 20 '25 20:11 sdetweil

There is no communication from the event source (google, iCloud,…) immediately when an event ends We have to fetch the entire event list (yes 5 yr old ones too) and process them. Then filter to the number of events you want to see

sdetweil avatar Nov 20 '25 20:11 sdetweil

The fetchInterval property defines how often to get the iCal source from the provider

sdetweil avatar Nov 20 '25 20:11 sdetweil

I get most of your points, but when the bug triggers, I can see the same calendar sending updates with only a few seconds interval.

I don't think it's fetching the data again, since I guess that would show on the terminal output.

michaelarnauts avatar Nov 21 '25 17:11 michaelarnauts

correct, it should show in the log of the message output from npm start

sdetweil avatar Nov 21 '25 18:11 sdetweil

Screenshot from the websocket messages. Take a look at the timings. This is a filter on only one agenda, the other 6 agenda's are also showing this behaviour.

Image

michaelarnauts avatar Nov 23 '25 11:11 michaelarnauts

The log output is only showing the "Use existing calendarfetcher for url" for all agendas for every pageload.

michaelarnauts avatar Nov 23 '25 11:11 michaelarnauts

@michaelarnauts Would it be possible for you to test this on the develop branch as well to see if it behaves the same there? There have been a few changes to the calendar module, but I can't say whether they might affect this.

KristjanESPERANTO avatar Nov 23 '25 12:11 KristjanESPERANTO

here is how to get the develop branch https://forum.magicmirror.builders/topic/14327/testing-new-fixes-or-solving-current-problems-with-next-release-code

sdetweil avatar Nov 25 '25 01:11 sdetweil

My log file is displaying this on the latest released version.

[2025-11-29 11:07:35.025] [ERROR] Calendar Error. Could not fetch calendar:  https://calendar.google.com/calendar/ical/nl.be%23holiday%40group.v.calendar.google.com/public/basic.ics Error: Too Many Requests
    at NodeHelper.checkFetchStatus (/opt/magic_mirror/js/node_helper.js:102:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[2025-11-29 11:07:35.283] [ERROR] Calendar Error. Could not fetch calendar:  https://calendar.google.com/calendar/ical/xxx%40gmail.com/private-xxx/basic.ics Error: Too Many Requests
    at NodeHelper.checkFetchStatus (/opt/magic_mirror/js/node_helper.js:102:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[2025-11-29 11:07:36.290] [ERROR] Calendar Error. Could not fetch calendar:  https://calendar.google.com/calendar/ical/xxx%40gmail.com/private-xxx/basic.ics Error: Too Many Requests
    at NodeHelper.checkFetchStatus (/opt/magic_mirror/js/node_helper.js:102:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[2025-11-29 11:07:36.778] [ERROR] Calendar Error. Could not fetch calendar:  https://calendar.google.com/calendar/ical/xxx%40gmail.com/private-xxx/basic.ics Error: Too Many Requests
    at NodeHelper.checkFetchStatus (/opt/magic_mirror/js/node_helper.js:102:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[2025-11-29 11:07:37.734] [ERROR] Calendar Error. Could not fetch calendar:  https://calendar.google.com/calendar/ical/xxx%40gmail.com/private-xxx/basic.ics Error: Too Many Requests
    at NodeHelper.checkFetchStatus (/opt/magic_mirror/js/node_helper.js:102:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[2025-11-29 11:07:40.388] [ERROR] Calendar Error. Could not fetch calendar:  https://calendar.google.com/calendar/ical/xxx%40gmail.com/private-xxx/basic.ics Error: Too Many Requests
    at NodeHelper.checkFetchStatus (/opt/magic_mirror/js/node_helper.js:102:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[2025-11-29 11:07:42.599] [ERROR] Calendar Error. Could not fetch calendar:  https://calendar.google.com/calendar/ical/xxx%40gmail.com/private-xxx/basic.ics Error: Too Many Requests
    at NodeHelper.checkFetchStatus (/opt/magic_mirror/js/node_helper.js:102:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

I'm using the docker image karsten13/magicmirror, but there is a :develop tag that tracks the develop branch. I'll try that and hope the rate limit solves itself soon.

michaelarnauts avatar Nov 29 '25 10:11 michaelarnauts

I'm still getting that error now on the latest develop branch, google is refusing to give me the ics file.

Navigating in the browser gives me a captcha that I need to solve, but that only fixes it for the browser, not for magicmirror.

michaelarnauts avatar Nov 29 '25 10:11 michaelarnauts

@michaelarnauts Can you update your develop branch and test it again? After merging #3976, it should now work 🙂

KristjanESPERANTO avatar Dec 08 '25 09:12 KristjanESPERANTO

I'll wait untill the karsten13/magicmirror:develop docker image has been rebuild (it should do so twice per day).

michaelarnauts avatar Dec 08 '25 16:12 michaelarnauts

I've updated my instance of magicmirror and will follow up in the course of the following days

michaelarnauts avatar Dec 09 '25 13:12 michaelarnauts

So far, it's going okay.

I do wonder however if it should be considered a bug that when one browser starts magicmirror (the screensaver starts on a wall mounted tablet), all other open browsers that are already showing magicmirror show a fadeout and fadein of the calendar because CALENDAR_EVENTS are broadcasted again.

This is okay when using magic mirror on one device, but when running on multiple, all devices flash when one device starts magicmirror again.

I guess this could be solved by only refreshing the calendar when something has actually changed? We could keep a hash or something in memory?

michaelarnauts avatar Dec 09 '25 16:12 michaelarnauts

well, there are three problems..

  1. THIS browser is just starting, so EVERYTHING has changed...
  2. the communications process sends ONE message, and ALL clients get it at the same time. we don't KNOW how many clients are attached..(each module instance is a client, all modules in the same system, or across client systems too)
  3. as the config file is identical on all browsers we have no way to differentiate requests across systems... (using identifier is identical across systems)..

i did create a small module(server resident) a couple years ago, to allow clients to get their ip address in the browser and use it as part of the identifier, then the broadcast back could be differentiated at the browser side when the data comes back. we currently have no code to do this differentiation

the current module design is one fetcher per URL (server resident) for all clients, started once, sent asynchronously..,

sdetweil avatar Dec 09 '25 17:12 sdetweil

Ah, you are thinking of the receiver side We could calculate a crc I guess to keep the storage requirements down

We have already broadcast them to other modules

sdetweil avatar Dec 09 '25 19:12 sdetweil