z_controller_helper does not consider json encoding

[heiheshang]

maybe I'm wrong, but from_json does not account for the encoding in the header {Body, Context1} = req_body(Context), Data = jsxrecord:decode(Body), {Data, Context1}. I get data in utf-8 and I have to convert it myself since there is Cyrillic

[mworrell]

It should just accept utf8 as the character set:

(zotonic@PoToi)1> jsxrecord:decode(<<"{ \"a\":\"éüç\" }"/utf8>>).
#{<<"a">> => <<"éüç"/utf8>>}

We don't accept any other character set.

[mworrell]

Can you give an example of your data?

[heiheshang]

#{<<"agreement">> => <<>>,<<"primer">> => <<208,191,209,128,208,19 0,208,178,208,181,209,128,208,186,208,176>>,<<"queryText">> => <<"да">>}

[heiheshang]

unicode:characters_to_list(maps:get(<<"queryText">>,Payload)) I do the conversion myself in my code

[mworrell]

Was the queryText coming from the user or from Erlang?

If you use Erlang then you have to add /utf8 to the binary, like this: <<"Grün"/utf8>>

[mworrell]

Afaik unicode:characters_to_list/1 consumes iodata with unicode (alias for utf8) encoding and translates it to a list with unicode code points. Might it be that you need a list and not an utf-8 binary?

[heiheshang]

message comes from another program

[heiheshang]

I want to make a match << "Да" >> with << ""да"" >>

[mworrell]

<<"Да"/utf8>> is equal to <<208,148,208,176>>.

I think the <<"да">> you are seeing is actually the latin1 representation of <<208,148,208,176>>