noobaa-core icon indicating copy to clipboard operation
noobaa-core copied to clipboard

[NSFS] Fix Newline Reader to work with partial reads and improve its memory usage

Open tangledbytes opened this issue 4 months ago • 0 comments

Explain the changes

This PR intends to fix the parsing issue present in our NewLine reader. Previously, the reader would perform partial reads (4096 bytes) and would force covert it into string, this works most of the times but falls apart if there was a UTF8 character at the boundary of the 4096 buffer.

This PR removes any such conversions while scanning through the buffer and also ensures to efficiently use the buffer, doesn't creates unnecessary strings and buffers and rather tries to rely on a pre-allocated buffers.

Issues: Fixed #xxx / Gap #xxx

  1. A DBS3 issue

Testing Instructions:

  1. ./node_modules/.bin/jest src/test/unit_tests/jest_tests/test_newline_reader.test.js
  • [ ] Doc added/updated
  • [ ] Tests added

tangledbytes avatar Oct 10 '24 16:10 tangledbytes