BUG: Cannot read properties of null (reading 'stream')

Open Llorx opened this issue 3 months ago • 0 comments

I was inserting data on a database where the disk is full, so it kept returning error: No space left on device.

When I receive an error, I retry the insert 5 seconds later.

After multiple retries, I ended receiving this unhandled error:

TypeError: Cannot read properties of null (reading 'stream'). Stack:
    at CopyStreamQuery.flushChunk (.../modules/Historian/node_modules/pg-copy-streams/copy-from.js:120:21)
    at CopyStreamQuery.flush (.../modules/Historian/node_modules/pg-copy-streams/copy-from.js:101:17)
    at CopyStreamQuery._writev (.../modules/Historian/node_modules/pg-copy-streams/copy-from.js:57:19)
    at doWrite (node:internal/streams/writable:596:12)
    at clearBuffer (node:internal/streams/writable:775:5)
    at onwrite (node:internal/streams/writable:653:7)
    at Object.onceWrapper (node:events:632:28)
    at Socket.emit (node:events:518:28)
    at afterWrite (node:internal/streams/writable:705:12)
    at onwrite (node:internal/streams/writable:688:7)

And it is an unhandled error. This drops the entire program.

The retry loop looks like so:

while (true) {
  try {
    const client = await pool.connect();
    const copyQuery = "COPY tag_history (ts, tag_id, quality, value) FROM STDIN BINARY";
    const stream = client.query(copyFrom(copyQuery));
    const readable = new Readable();
    // [...] Insert all the bytes in the readable
    readable.pipe(stream);
    await new Promise((resolve, reject) => {
        stream.on("finish", resolve);
        stream.on("error", reject);
    });
    break;
  } catch (e) {
    console.warn(e);
    await setTimeout(5000);
  }
}

It seems that the connection closed (or something) during the write process, invalidating an internal stream to null but there's a race condition where the code keeps accessing the internal stream.

We are going to disable pg-copy-stream as it is not production ready. We cannot let the app to suddenly die.

The log looks like so:

05:01:40.158 | INFO  | TimescalePool -> Bulk #1 reinserting 30000 entries
05:01:40.227 | INFO  | TimescalePool -> Bulk #5 reinserting 30000 entries
05:01:40.234 | WARN  | TimescalePool -> Error inserting bulk #1: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:40.234 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:40.290 | WARN  | TimescalePool -> Error inserting bulk #5: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:40.290 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:43.815 | INFO  | TimescalePool -> Bulk #2 reinserting 30000 entries
05:01:43.879 | WARN  | TimescalePool -> Error inserting bulk #2: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:43.879 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:44.089 | INFO  | TimescalePool -> Bulk #4 reinserting 10000 entries
05:01:44.092 | INFO  | TimescalePool -> Bulk #3 reinserting 30000 entries
05:01:44.174 | WARN  | TimescalePool -> Error inserting bulk #4: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:44.174 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:44.174 | WARN  | TimescalePool -> Error inserting bulk #3: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:44.174 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:45.234 | INFO  | TimescalePool -> Bulk #1 reinserting 30000 entries
05:01:45.315 | INFO  | TimescalePool -> Bulk #5 reinserting 30000 entries
05:01:45.321 | WARN  | TimescalePool -> Error inserting bulk #1: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:45.321 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:45.380 | WARN  | TimescalePool -> Error inserting bulk #5: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:45.380 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:48.881 | INFO  | TimescalePool -> Bulk #2 reinserting 30000 entries
05:01:48.969 | WARN  | TimescalePool -> Error inserting bulk #2: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:48.969 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:49.175 | INFO  | TimescalePool -> Bulk #4 reinserting 10000 entries
05:01:49.179 | INFO  | TimescalePool -> Bulk #3 reinserting 30000 entries
05:01:49.197 | WARN  | TimescalePool -> Error inserting bulk #4: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:49.197 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:49.264 | WARN  | TimescalePool -> Error inserting bulk #3: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:49.264 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:50.322 | INFO  | TimescalePool -> Bulk #1 reinserting 30000 entries
05:01:50.385 | INFO  | TimescalePool -> Bulk #5 reinserting 30000 entries
05:01:50.391 | WARN  | TimescalePool -> Error inserting bulk #1: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:50.391 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:50.450 | WARN  | TimescalePool -> Error inserting bulk #5: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:50.450 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:53.973 | INFO  | TimescalePool -> Bulk #2 reinserting 30000 entries
05:01:53.994 | WARN  | TimescalePool -> Error inserting bulk #2: error: could not extend file "base/136550600/136553251" with FileFallocate(): No space left on device
05:01:53.994 | INFO  | TimescalePool -> Retrying in 5000 ms
05:01:53.995 | ERROR | main -> [FATAL ERROR] TypeError: Cannot read properties of null (reading 'stream'). Stack: 
    at CopyStreamQuery.flushChunk (.../modules/Historian/node_modules/pg-copy-streams/copy-from.js:120:21)
    at CopyStreamQuery.flush (.../modules/Historian/node_modules/pg-copy-streams/copy-from.js:101:17)
    at CopyStreamQuery._writev (.../modules/Historian/node_modules/pg-copy-streams/copy-from.js:57:19)
    at doWrite (node:internal/streams/writable:596:12)
    at clearBuffer (node:internal/streams/writable:775:5)
    at onwrite (node:internal/streams/writable:653:7)
    at Object.onceWrapper (node:events:632:28)
    at Socket.emit (node:events:518:28)
    at afterWrite (node:internal/streams/writable:705:12)
    at onwrite (node:internal/streams/writable:688:7)

Oct 17 '25 07:10 Llorx