go-openai icon indicating copy to clipboard operation
go-openai copied to clipboard

EscapeString

Open rhinogo opened this issue 2 years ago • 4 comments

The official Python package, when converting objects to JSON strings, will convert Chinese and other text into Unicode encoding, and can effectively avoid sensitive words in responses. This modification is to enable the Golang package to have this capability.

rhinogo avatar Jul 14 '23 05:07 rhinogo

Hey, thank you for the PR! Could you please provide an illustration where default Go's string handling doesn't work? And what could be our alternatives here?

sashabaranov avatar Jul 16 '23 15:07 sashabaranov

when there are Emoji in string, Marshal would fail:

json: error calling MarshalJSON for type openai.EscapeString: invalid character 'U' in string escape code

for example, 😗 will be quoted to ASCII \U0001f617 , but it is not valid in json (encoding/json/scanner.go:353). Because of the \U

codingknees avatar Jul 26 '23 12:07 codingknees

when there are Emoji in string, Marshal would fail:

json: error calling MarshalJSON for type openai.EscapeString: invalid character 'U' in string escape code

for example, 😗 will be quoted to ASCII \U0001f617 , but it is not valid in json (encoding/json/scanner.go:353). Because of the \U

I find that in python, json.dumps encode that Emoji to \ud83d\ude17. So strange

codingknees avatar Jul 26 '23 12:07 codingknees

hack as ` EscapeString string

func (esc EscapeString) MarshalJSON() ([]byte, error) { x := strings.ReplaceAll(strconv.QuoteToASCII(string(esc)), "\U", "\u") return []byte(x), nil }`

codingknees avatar Jul 26 '23 13:07 codingknees