Calendar loading randomly fails if files stored locally

[drtorchwood]

Platform: Raspberry PI 4 (bullseye)

Node Version: 20.5.1

MagicMirror² Version: 2.26.0

Description: I have several calendars stored locally and they are set in the config file as: url: "webcal://0.0.0.0:8090/modules/calendar_files/geburtstage.ics" and fetchInterval: 120000. The files only change once per day. However, I often receive messages that the calendar cannot be loaded. A typical error message is:

[18.01.2024 20:02.38.254] [ERROR] Calendar Error. Could not fetch calendar:  http://0.0.0.0:8090/modules/calendar_files/geburtstage.ics TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11576:11) {
  cause: Error: read ECONNRESET
      at TCP.onStreamRead (node:internal/stream_base_commons:217:20) {
    errno: -104,
    code: 'ECONNRESET',
    syscall: 'read'
  }
}

A few minutes later the calendar can be loaded without any change to the file. Sometimes one of the other calendars fail. I changed the log level to DEBUG but there is no additional information at that time. The previous message comes from another calendar file:

[18.01.2024 20:02.38.164] [INFO]  Calendar-Fetcher: Broadcasting 50 events from http://0.0.0.0:8090/modules/calendar_files/privat.ics.

The next message is related to a third file:

[18.01.2024 20:02.38.518] [DEBUG] parsed data={"8b65f88b-4983-40f1-9060-898f88c31eb5":{

If I don't load the biggest calendar file (~500KB), the error does not occur. Putting the big file at the end of the files does not help.

Summary: The configuration is correct and the calendar files are valid. If multiple are loaded and there is a bigger one (~500KB), it fails randomly.

[khassel]

• is this new with v2.26.0 ?
• are you using npm start (electron) or npm run server (server only) for starting?

[sdetweil]

in calendar config, please use localhost instead of 0.0.0.0, which means any/all interfaces @drtorchwood

[drtorchwood]

in calendar config, please use localhost instead of 0.0.0.0, which means any/all interfaces @drtorchwood

I followed this approach (https://github.com/MagicMirrorOrg/MagicMirror/issues/3147) but will try with 127.0.0.1 later.

[drtorchwood]

• is this new with v2.26.0 ?
• are you using npm start (electron) or npm run server (server only) for starting?

I'm using npm start. The problem also occurred in v2.25.0 but in a previous version it was working (I don't know which one). However, the calendar file might have been slightly smaller at that time.

I will try to revert to a previous version or shorten the file slightly. Hopefully, I can try this today evening and will report then.

[khassel]

we have already other problems with the node version which is used by electron v27.2.0 (which is used in the v2.26.0 release).

So you could test with a newer elctron version with npm install [email protected]

[drtorchwood]

in calendar config, please use localhost instead of 0.0.0.0, which means any/all interfaces @drtorchwood

I followed this approach (#3147) but will try with 127.0.0.1 later.

we have already other problems with the node version which is used by electron v27.2.0 (which is used in the v2.26.0 release).

So you could test with a newer elctron version with npm install [email protected]

I tested both proposals (127.0.0.1 and npm install [email protected]) but they do not solve the issue.

In the meantime, realized that the first time the error occurs, it is slightly different:

[22.01.2024 18:25.23.892] [ERROR] Calendar Error. Could not fetch calendar:  http://127.0.0.1:8090/modules/calendar_files/feiertage2.ics TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11576:11) {
  cause: SocketError: other side closed
      at Socket.onSocketEnd (node:internal/deps/undici/undici:9790:26)
      at Socket.emit (node:events:526:35)
      at endReadableNT (node:internal/streams/readable:1359:12)
      at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
    code: 'UND_ERR_SOCKET',
    socket: {
      localAddress: '127.0.0.1',
      localPort: 58200,
      remoteAddress: undefined,
      remotePort: undefined,
      remoteFamily: undefined,
      timeout: undefined,
      bytesWritten: 492,
      bytesRead: 441164
    }
  }
}

After that, the messages for every refresh is (as mentioned in the initial post):

[22.01.2024 18:27.30.243] [ERROR] Calendar Error. Could not fetch calendar:  http://127.0.0.1:8090/modules/calendar_files/feiertage2.ics TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11576:11) {
  cause: Error: read ECONNRESET
      at TCP.onStreamRead (node:internal/stream_base_commons:217:20) {
    errno: -104,
    code: 'ECONNRESET',
    syscall: 'read'
  }
}

I reverted to v2.24.0 (using git checkout e87f50e (hope that this is correct)) and the error has not occurred for several hours. I will wait for some more hours to be sure. Then, I will try to identify the commit that causes the error. From the history, this one (https://github.com/MagicMirrorOrg/MagicMirror/commit/f2957f90df66a2392b190e68a923f97c9087892b) would be my starting point.

[sdetweil]

yes, we know that all that junk is coming from the internal fetch, but we don't know WHY.. this was supposed to get RID of problems.. internal fetch has been around for years and electron just picked it up recently.

[khassel]

I reverted to v2.24.0 (using git checkout e87f50e (hope that this is correct)) and the error has not occurred for several hours. I will wait for some more hours to be sure. Then, I will try to identify the commit that causes the error. From the history, this one (https://github.com/MagicMirrorOrg/MagicMirror/commit/f2957f90df66a2392b190e68a923f97c9087892b) would be my starting point.

you are right so far:

• we replaced node-fetch with internal fetch provided by nodejs with https://github.com/MagicMirrorOrg/MagicMirror/commit/f2957f90df66a2392b190e68a923f97c9087892b
• the errors you are posting above are coming from undici which is used by internal fetch
• the internal fetch is experimental in nodejs v18 and official since v20
• electron is still using nodejs v18, see https://releases.electronjs.org/, so maybe this is fixed in a future electron version when using a newer nodejs version
• I don't think we will revert back to node-fetch

Edit: Would be interesting to test the upcoming electron version v29 npm i [email protected] because this contains nodejs v20

[khassel]

is this still an issue with latest MagicMirror Release v2.28.0?

Background: In the new release we are using electron v31 which internally uses node v20 (instead of v18 before).

[khassel]

seems to be solved with latest mm/electron version.