node icon indicating copy to clipboard operation
node copied to clipboard

stream: use ByteLengthQueuingStrategy when not in object mode

Open CGQAQ opened this issue 1 year ago • 23 comments

Use ByteLengthQueuingStrategy when not in object mode

After this PR

Array buffers memory usage is 0 MiB Array buffers memory usage is 0 MiB converting node stream to web stream Array buffers memory usage is 0 MiB Array buffers memory usage is 0 MiB Array buffers memory usage is 0 MiB Array buffers memory usage is 0 MiB Array buffers memory usage is 0 MiB Array buffers memory usage is 0 MiB Array buffers memory usage is 0 MiB reading the chunks Array buffers memory usage is 2 MiB Array buffers memory usage is 31 MiB Array buffers memory usage is 9 MiB Array buffers memory usage is 18 MiB Array buffers memory usage is 9 MiB Array buffers memory usage is 5 MiB Array buffers memory usage is 8 MiB Array buffers memory usage is 17 MiB Array buffers memory usage is 13 MiB Array buffers memory usage is 12 MiB Array buffers memory usage is 19 MiB Array buffers memory usage is 20 MiB Array buffers memory usage is 31 MiB Array buffers memory usage is 11 MiB Array buffers memory usage is 24 MiB Array buffers memory usage is 17 MiB

fixes: #46347

CGQAQ avatar Jul 20 '23 07:07 CGQAQ

This would probably not be spec compliant, https://streams.spec.whatwg.org/#readable-stream-default-controller-should-call-pull I do not think we are allowed to add additional steps not mentioned in the spec

debadree25 avatar Jul 20 '23 09:07 debadree25

should I remove the strategy.size ?

CGQAQ avatar Jul 21 '23 04:07 CGQAQ

A previous PR had attempted the same, nonethless would cc @nodejs/whatwg-stream to take a look!

debadree25 avatar Jul 22 '23 18:07 debadree25

This would probably not be spec compliant, streams.spec.whatwg.org/#readable-stream-default-controller-should-call-pull I do not think we are allowed to add additional steps not mentioned in the spec

Given that this is for our Node.js stream adapters, this change should be fine.

jasnell avatar Sep 14 '23 20:09 jasnell

CI: https://ci.nodejs.org/job/node-test-pull-request/53954/

nodejs-github-bot avatar Sep 14 '23 20:09 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/53958/

nodejs-github-bot avatar Sep 15 '23 07:09 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/54174/

nodejs-github-bot avatar Sep 23 '23 08:09 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/54250/

nodejs-github-bot avatar Sep 26 '23 14:09 nodejs-github-bot

Seemingly related CI failures:

not ok 690 parallel/test-stream-readable-to-web
  ---
  duration_ms: 172.99900
  severity: fail
  exitcode: 1
  stack: |-
    node:events:492
          throw er; // Unhandled 'error' event
          ^
    
    Error: ENOENT: no such file or directory, open 'C:\dev\urandom'
    Emitted 'error' event on ReadStream instance at:
        at emitErrorNT (node:internal/streams/destroy:151:8)
        at emitErrorCloseNT (node:internal/streams/destroy:116:3)
        at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
      errno: -4058,
      code: 'ENOENT',
      syscall: 'open',
      path: 'C:\\dev\\urandom'
    }
    
    Node.js v21.0.0-pre

aduh95 avatar Sep 30 '23 21:09 aduh95

Any updates on this?

idranme avatar Jan 22 '24 13:01 idranme

CI is failing

mcollina avatar Jan 22 '24 14:01 mcollina

All five failing tests are not found in ci.nodejs.org

image

CGQAQ avatar Jan 23 '24 00:01 CGQAQ

@mcollina All tests now passed, PTAL

CGQAQ avatar Jan 23 '24 04:01 CGQAQ

Let's see on our CI

mcollina avatar Jan 23 '24 06:01 mcollina

CI: https://ci.nodejs.org/job/node-test-pull-request/56894/

nodejs-github-bot avatar Jan 23 '24 07:01 nodejs-github-bot

What are these mean???? I don't understand what happend link

15:29:20 Counting node-test-binary-windows-native-suites. enabledIndex=2
15:29:20 node-test-binary-windows-js-suites: Quiet period groovy=[index < 5 ? 0 : 2 * 60], index=1 -> quietPeriodGroovy=0
15:29:20 quiet period for node-test-binary-windows-js-suites is 0 seconds.node-test-binary-windows-native-suites: Quiet period groovy=[index < 5 ? 0 : 2 * 60], index=2 -> quietPeriodGroovy=0
15:29:20 quiet period for node-test-binary-windows-native-suites is 0 seconds.Finished Build : [#25539](https://ci.nodejs.org/job/node-test-binary-windows-js-suites/25539//) of Job : [node-test-binary-windows-js-suites](https://ci.nodejs.org/job/node-test-binary-windows-js-suites/) with status : [FAILURE](https://ci.nodejs.org/job/node-test-binary-windows-js-suites/25539//console) at 02:41:03
15:41:30 Finished Build : [#21239](https://ci.nodejs.org/job/node-test-binary-windows-native-suites/21239//) of Job : [node-test-binary-windows-native-suites](https://ci.nodejs.org/job/node-test-binary-windows-native-suites/) with status : [SUCCESS](https://ci.nodejs.org/job/node-test-binary-windows-native-suites/21239//console) at 02:41:30
15:41:30 Build step 'MultiJob Phase' marked build as failure
15:41:31 Collecting metadata...
15:41:31 Metadata collection done.
15:41:31 Notifying upstream projects of job completion
15:41:31 Finished: FAILURE

CGQAQ avatar Jan 23 '24 09:01 CGQAQ

CI: https://ci.nodejs.org/job/node-test-pull-request/56896/

nodejs-github-bot avatar Jan 23 '24 11:01 nodejs-github-bot

What are these mean???? I don't understand what happend link

This is just some internal CI logic for running tests. What happened in the CI run is that some tests (flaky tests) failed on Windows and FreeBSD also failed. I've retriggered the CI for failed jobs.

StefanStojanovic avatar Jan 23 '24 11:01 StefanStojanovic

Same as before, what do you think? @mcollina

CGQAQ avatar Jan 24 '24 00:01 CGQAQ

CI: https://ci.nodejs.org/job/node-test-pull-request/56932/

nodejs-github-bot avatar Jan 26 '24 22:01 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/57008/

nodejs-github-bot avatar Feb 01 '24 14:02 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/57011/

nodejs-github-bot avatar Feb 02 '24 07:02 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/57012/

nodejs-github-bot avatar Feb 02 '24 09:02 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/59104/

nodejs-github-bot avatar May 11 '24 14:05 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/59146/

nodejs-github-bot avatar May 12 '24 07:05 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/59171/

nodejs-github-bot avatar May 12 '24 12:05 nodejs-github-bot

Landed in 261e88e269f6ddf285d05671a0cba7c2ed93e38a

aduh95 avatar May 12 '24 17:05 aduh95