go-openai
go-openai copied to clipboard
EscapeString
The official Python package, when converting objects to JSON strings, will convert Chinese and other text into Unicode encoding, and can effectively avoid sensitive words in responses. This modification is to enable the Golang package to have this capability.
Hey, thank you for the PR! Could you please provide an illustration where default Go's string handling doesn't work? And what could be our alternatives here?
when there are Emoji in string, Marshal would fail:
json: error calling MarshalJSON for type openai.EscapeString: invalid character 'U' in string escape code
for example, 😗 will be quoted to ASCII \U0001f617 , but it is not valid in json (encoding/json/scanner.go:353). Because of the \U
when there are Emoji in string, Marshal would fail:
json: error calling MarshalJSON for type openai.EscapeString: invalid character 'U' in string escape codefor example, 😗 will be quoted to ASCII \U0001f617 , but it is not valid in json (encoding/json/scanner.go:353). Because of the \U
I find that in python, json.dumps encode that Emoji to \ud83d\ude17. So strange
hack as ` EscapeString string
func (esc EscapeString) MarshalJSON() ([]byte, error) { x := strings.ReplaceAll(strconv.QuoteToASCII(string(esc)), "\U", "\u") return []byte(x), nil }`