encoding_rs_io

encoding_rs_io copied to clipboard

Currently `rg` with `--multiline` when operating on many files can be 50x slower without `--mmap` compared to `--multiline --mmap`. More than 99% of CPU time is spent in `ReadBuf::initialize_unfilled`, which...

icecream95

UTF8 to Any Encoder implementation

10

Is there anything in progress/alternative regarding the first point of the Future work paragraph of the README ? > An encoder that accepts an arbitrary std::io::Write implementation and takes valid...

artem-tim

dralley

This implements a `into_inner` method that returns the inner buffer, like `BufReader` does. My use case for this was needing to seek the inner reader, I tried implementing `Seek` but...

boudra

[RFE] User-provided encoding detection function

1

As an example, the XML specification recommends a special encoding detection scheme in cases where the BOM doesn't exist: https://www.w3.org/TR/xml11/#sec-guessing In the event that a single-byte, ascii-compatible encoding is being...

dralley

…rences

atouchet

[RFE] Asynchronous IO support

5

Would you be open to a feature that supported Tokio's `AsyncRead` trait for the underlying IO?

dralley

Always transcode Utf8

2

I'm using encoding_rs_io to make a stream of always valid utf8 because invalid utf8 is not handled upstream. The way the options are laid out at present it seems there's...

That3Percent

This is a possibly somewhat common method, and the decoder in this crate could implement it more efficiently by avoiding UTF-8 re-validation. See @hsivonen's comment here: https://github.com/hsivonen/encoding_rs/issues/8#issuecomment-411356371

BurntSushi