Memory leak when reading data from Twinkie device

[RajaaAbdallah]

SerialPort Version

11.0.0

Node Version

16.13.2

Electron Version

23.3.7

Platform

Linux user-LIFEBOOK-E559 5.4.0-148-generic #165~18.04.1-Ubuntu SMP Thu Apr 20 01:14:06 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Architecture

x64

Hardware or chipset of serialport

No response

What steps will reproduce the bug?

We are receiving real-time data from the Twinkie device, and after continuously reading the data for two hours, we encountered a memory leak.

The Twinkie v2 device is connected to a laptop via a USB wire, and we utilize the serial port to retrieve the data. However, a memory leak occurs after two hours of continuous data reading

this is the snippet code we using it to reading real-time data: First, the open() method is responsible for opening the serial port connected to the device: open(): Promise<void> { return new Promise<void>(async (resolve, reject) => { const port = new SerialPort( { path: this.sessionInfo.path, baudRate: this.sessionInfo.buadRate, dataBits: 8, stopBits: 2, parity: "none", }, function (err) { if (err) { reject(err); } } ); this.dataReaderParser = this.sessionInfo.readLineParser; port.pipe(this.bufferCleanerParser); this.bufferCleanerParser.on("data", (data: Buffer) => { console.log(Cleared data: ${data.toString("hex")}); }); await new Promise<void>((resolve) => { setTimeout(() => { port.unpipe(this.bufferCleanerParser); port.pipe(this.dataReaderParser); resolve(); }, 10); }); const onData = (data: string) => { if (this.onDataCallback) { this.onDataCallback(data); } this.emit('data', data); }; this.dataReaderParser.on('data', onData); this.port = port; resolve(); }); } Next, the read() method is used to listen for emitted data:

public read( callback: (data: string) => void, sendFlag?: boolean ): Promise<string> { if (callback) { return new Promise<string>((resolve, reject) => { if (sendFlag) { this.onDataCallback = (data: string) => { callback(data); }; return; } this.on("data", (data) => { callback(data); }); this.removeListener('data', callback) }); } return Promise.resolve("the callback is not provided"); }

Finally, the readAnalog() method is called when the start button is pressed on the UI to continuously read data from the connected device: async readAnalog(decoder: DecoderInterface, callback: (data: PowerDeliveryAnalogModel|any) => void) { try { this.writeDigital() const session = this.sessionManager.getSession("serialPortSession2"); session.read(async (data: string) => { try { this.counter++; if (this.counter === 50) { const parserAnalogData = this.parser.parseAnalogData(data); const res: PowerDeliveryAnalogModel = decoder.decodeAnalogData(parserAnalogData); callback(res); this.counter = 0; } } catch (error) {} }); } catch (error) {} }

so how we can avoid memory leaks that occur by serial port listeners?

I think my issue is similar to this issue related to a memory leak when reading GPS data

What happens?

After continuously reading real-time data from the Twinkie v2 device for a duration of two hours, here is the result it provided:

[electron] <--- Last few GCs ---> [electron] [electron] [11247:0x3aa8002c0000] 4812376 ms: Scavenge 10.4 (20.9) -> 4.2 (20.9) MB, 0.13 / 0.00 ms (average mu = 1.000, current mu = 1.000) task; [electron] [11247:0x3aa8002c0000] 4812823 ms: Scavenge 10.4 (20.9) -> 4.2 (20.9) MB, 0.13 / 0.00 ms (average mu = 1.000, current mu = 1.000) task; [electron] [11247:0x3aa8002c0000] 4813281 ms: Scavenge 10.4 (20.9) -> 4.2 (20.9) MB, 0.19 / 0.00 ms (average mu = 1.000, current mu = 1.000) task; [electron] [electron] [electron] <--- JS stacktrace ---> [electron] [electron] FATAL ERROR: Cannot grow ExternalPointerTable past its maximum capacity Allocation failed - process out of memory

What should have happened?

If the bug is resolved, it should fix the memory leak error that occurs as a result of the serial port listener. Once the issue is resolved, the application should be able to continuously read real-time data for days without encountering the memory leak exception, thereby preventing the application from stopping.

Additional information

No response

[GazHank]

If I'm reading the code snippets correctly I think you are continually adding (and removing) event listeners. Would it not be possible to add the on data event listener when the port is opened, and retain that listener for the whole time that the port is open, rather that adding the listener every time the data is written?

[RajaaAbdallah]

Regarding your question about "continually adding (and removing) event listeners", I only add it once.

It will cause the same problem if you use this simple code to listen to a serial port that continuously sending data.

import { SerialPort } from "serialport";
  const port = new SerialPort({
    path: "/dev/ttyACM1",
    baudRate: 9600,
    dataBits: 8,
    stopBits: 2,
    parity: "none",
  });

  port.on('data', (data: Buffer) => {
    // const receivedData = data.toString();
    // console.log('Received data:', receivedData);
  });

  port.on("open", () => {
    console.log("Serial port is open");
  });

Example of memory usage while reading data:

After leaving it for 3 min and stopping sending data, the memory is still reserved :

[GazHank]

My mistake it thought you were adding the lister each time you were writing data

[RajaaAbdallah]

any update, please?

[GazHank]

In your simplified example code, are you able to confirm if they continue to see Electron growing continually if you leave it running for longer. When I try this I get a stable memory usage (with some minor variation at times as the garbage collector does it's thing).

So at the moment I'm unable to recreate the issue you are seeing