misskey icon indicating copy to clipboard operation
misskey copied to clipboard

perf: Use stream around MediaProxy and FileServer

Open tamaina opened this issue 2 years ago • 5 comments

Resolve #7711 Based on #9459

What

Why

Additional info (optional)

tamaina avatar Jan 01 '23 18:01 tamaina

ReadableStream.tee() 相当のAPIがないとダメだわ

tamaina avatar Jan 01 '23 18:01 tamaina

ERR  1  [core]  Error [ERR_STREAM_PUSH_AFTER_EOF]: stream.push() after EOF
Trace: Error [ERR_STREAM_PUSH_AFTER_EOF]: stream.push() after EOF
    at new NodeError (node:internal/errors:393:5)
    at readableAddChunk (node:internal/streams/readable:285:30)
    at Readable.push (node:internal/streams/readable:234:10)
    at node:internal/webstreams/adapters:496:22
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ERR_STREAM_PUSH_AFTER_EOF'
}
    at process.<anonymous> (file:///home/aqz/misskey/packages/backend/built/boot/index.js:43:17)
    at process.emit (node:events:525:35)
    at process.emit (node:domain:489:12)
    at process._fatalException (node:internal/process/execution:149:25)

が出てくる

tamaina avatar Jan 07 '23 04:01 tamaina

Codecov Report

Merging #9453 (cdef862) into develop (2c52655) will decrease coverage by 0.09%. The diff coverage is 22.28%.

:exclamation: Current head cdef862 differs from pull request most recent head cb2823d. Consider uploading reports for the commit cb2823d to get more accurate results

@@             Coverage Diff             @@
##           develop    #9453      +/-   ##
===========================================
- Coverage    22.72%   22.63%   -0.09%     
===========================================
  Files          734      731       -3     
  Lines        68747    68609     -138     
  Branches      2028     2033       +5     
===========================================
- Hits         15620    15527      -93     
+ Misses       53127    53082      -45     
Impacted Files Coverage Δ
packages/backend/src/server/FileServerService.ts 0.00% <0.00%> (ø)
...ages/backend/src/server/MediaProxyServerService.ts 0.00% <0.00%> (ø)
packages/backend/src/core/DownloadService.ts 51.49% <26.47%> (-5.79%) :arrow_down:
packages/backend/src/core/FileInfoService.ts 55.77% <26.53%> (-4.09%) :arrow_down:
...ackages/backend/src/core/ImageProcessingService.ts 50.34% <48.64%> (+2.98%) :arrow_up:
packages/backend/src/logger.ts 62.29% <100.00%> (-1.35%) :arrow_down:
packages/backend/src/misc/secure-rndstr.ts 90.47% <0.00%> (-9.53%) :arrow_down:
packages/backend/src/core/RoleService.ts 36.27% <0.00%> (-0.31%) :arrow_down:
...ackages/backend/src/models/entities/UserProfile.ts 97.84% <0.00%> (-0.12%) :arrow_down:
... and 22 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Jan 07 '23 04:01 codecov[bot]

https://github.com/nodejs/node/pull/45026

これなんですかね

tamaina avatar Jan 07 '23 05:01 tamaina

p1.a9z.devで動作中

(.node-versionを>= 18.13.0 < 19.0.0 >= 19.4.0みたいにしたいところだけど流石に許容するソフトがないか?)

tamaina avatar Jan 15 '23 13:01 tamaina

コンフリクト解消が面倒になったのでclose

tamaina avatar Jan 25 '23 06:01 tamaina

というかこれ、NodeのWeb Streamが遅いらしくて、多分99%の場合は遅い
(重たい動画ファイルは別にして)

tamaina avatar Jan 25 '23 13:01 tamaina