encoding_rs icon indicating copy to clipboard operation
encoding_rs copied to clipboard

Encoding and decoding behavior difference

Open artem-tim opened this issue 2 years ago • 0 comments

Please help me understand whether the difference in behavior between Encoder and Decoder is intentional or not. Consider the following code :

let t_utf8 = "ハロー世界";
let t_iso_2022_jp = b"\x1B\x24\x42\x25\x4F\x25\x6D\x21\x3C\x40\x24\x33\x26\x1B\x28\x42";

{
    let mut dec = ISO_2022_JP.new_decoder();
    let mut dst = [0 as u8; 6];
    let out = dec.decode_to_utf8(t_iso_2022_jp, &mut dst, false);
    println!("Partially decoded : {:?}", dst);
    println!("Result : {:?}", out);
}

{
    let mut enc = ISO_2022_JP.new_encoder();
    let mut dst = [0 as u8; 6];
    let out = enc.encode_from_utf8(t_utf8, &mut dst, false);
    println!("Partially encoded : {:?}", dst);
    println!("Result : {:?}", out);
}

Here is the output :

Partially decoded : [227, 131, 143, 227, 131, 173]
Result : (OutputFull, 7, 6, false)
Partially encoded : [0, 0, 0, 0, 0, 0]
Result : (OutputFull, 0, 0, false)

In both cases, we are attempting to decode or encode something that wouldn't fit the target buffer size, however the decoder is able to partially fill the buffer, where the encoder just does nothing. If the buffer size is big enough, both work normally. Is this behavior expected ?

artem-tim avatar Mar 28 '23 12:03 artem-tim