deno_std icon indicating copy to clipboard operation
deno_std copied to clipboard

feat(streams): add `Base64EncoderStream` and `Base64DecoderStream`

Open babiabeo opened this issue 1 year ago • 6 comments

Close: #3087

babiabeo avatar May 16 '24 15:05 babiabeo

Codecov Report

Attention: Patch coverage is 92.85714% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 91.89%. Comparing base (377043c) to head (7aa7e50).

Files Patch % Lines
streams/base64_decoder_stream.ts 84.84% 5 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4756      +/-   ##
==========================================
- Coverage   91.89%   91.89%   -0.01%     
==========================================
  Files         485      488       +3     
  Lines       41308    41391      +83     
  Branches     5317     5321       +4     
==========================================
+ Hits        37959    38035      +76     
- Misses       3292     3299       +7     
  Partials       57       57              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar May 17 '24 04:05 codecov[bot]

As suggested by @BlackAsLight, I have removed line breaks handling from both encoder and decoder.

babiabeo avatar May 18 '24 10:05 babiabeo

As suggested by @BlackAsLight, I have removed line breaks handling from both encoder and decoder.

I think this also works. In this case, we can remove/insert line-breaks before/after the decoding/encoding.

kt3k avatar May 21 '24 04:05 kt3k

I wonder if it makes more sense to put these APIs next to the encodeBase64/decodeBase64 APIs in @std/encoding/base64. WDYT? cc @iuioiua @crowlKats

kt3k avatar May 21 '24 05:05 kt3k

I wonder if it makes more sense to put these APIs next to the encodeBase64/decodeBase64 APIs in @std/encoding/base64. WDYT? cc @iuioiua @crowlKats

Yep.

iuioiua avatar May 21 '24 06:05 iuioiua

Before merging this in, I'd like to know more about any justifications for encoding/decoding streams. Adding these APIs will likely set a precedent for other streaming modes like base 32, etc.

iuioiua avatar May 21 '24 06:05 iuioiua

I'd like to know more about any justifications for encoding/decoding streams.

It's used for encoding/decoding a huge data, the size of which exceeds the memory limit of your program.

kt3k avatar May 27 '24 09:05 kt3k

We recently agreed to not have encoding/decoding + streaming APIs in the Standard Library as we thought there aren't many use cases. Perhaps, we were wrong. If we say yes to this API, we should say yes to the rest.

iuioiua avatar May 27 '24 11:05 iuioiua

We recently agreed to not have encoding/decoding + streaming APIs in the Standard Library as we thought there aren't many use cases.

Can you link to the discussion?

kt3k avatar May 27 '24 15:05 kt3k

The agreement was made in a previous weekly deno_std meeting. I sent you the details.

To be clear, I'm happy with having Base64EncoderStream in the Standard Library; we should be aware that doing so will set a precedent and justify having the others in there, too. And at that point, maybe we should just include @doctor/encoding-stream in the Standard Library (with permission from and proper attribution for @BlackAsLight, of course).

iuioiua avatar May 27 '24 21:05 iuioiua

And at that point, maybe we should just include @doctor/encoding-stream in the Standard Library (with permission from and proper attribution for @BlackAsLight, of course).

Sure

BlackAsLight avatar May 28 '24 05:05 BlackAsLight

If encoding/decoding + streaming APIs are allowed in the Standard Library, should we create a new PR for all stream encoders/decoders and close this PR? This PR only implements the base64 encoder and decoder based on mazira/base64-stream which may not be the same as @doctor/encoding-stream/base64.

babiabeo avatar May 28 '24 16:05 babiabeo

This PR only implements the base64 encoder and decoder based on mazira/base64-stream which may not be the same as @doctor/encoding-stream/base64.

Your implementation is more or less the same as mine. I would characterise mine as more tidy though.

BlackAsLight avatar May 28 '24 18:05 BlackAsLight

Yes, let's close this PR and open a new one, adding @doctor/encoding-stream to std/encoding in new files. I.e., encoding/base32_stream.ts, etc.

iuioiua avatar May 28 '24 23:05 iuioiua