sext icon indicating copy to clipboard operation
sext copied to clipboard

Add option to choose encoding for non-ASCII atoms (latin1 or utf8)

Open seriyps opened this issue 1 year ago • 3 comments

This addresses the #40

However it doesn't add support to encode_prefix / decode_prefix, mainly because Legacy option was not supported there as well. However I think I should actually add it. Maybe separate commit? Readme should be probably updated as well.

I didn't run tests because I don't have quickcheck, but it compiles and seems to work:

1> sext:encode('привет').
** exception error: bad argument
     in function  atom_to_binary/2
        called as atom_to_binary('привет',latin1)
        *** argument 1: contains a character not expressible in latin1
     in call from sext:encode_atom/2 (/Users/sergey.prokhorov/workspace/sext/src/sext.erl, line 411)
2> sext:encode('привет', #{atom_encoding => utf8}).
<<12,232,111,250,56,14,134,227,161,178,232,109,122,56,32,8>>
3> sext:decode(sext:encode('привет', #{atom_encoding => utf8})).
'пÑ\200ивеÑ\202'
4> sext:decode(sext:encode('привет', #{atom_encoding => utf8}), #{atom_encoding => utf8}).
'привет'

So, what do you think?

seriyps avatar Oct 17 '24 10:10 seriyps

I think you need to handle the code being called with a (legacy) boolean() argument as well. Annoying, but won't be that hard.

uwiger avatar Oct 17 '24 11:10 uwiger

Done, good point!

seriyps avatar Oct 17 '24 12:10 seriyps

Thanks. Will try to test this during the weekend.

uwiger avatar Oct 25 '24 07:10 uwiger