FastChat
FastChat copied to clipboard
fix: garbled Chinese in the conv.json
Why are these changes needed?
I need save the battle logs in the conv.json, and then I found that garbled Chinese in the file. The reason is json.dumps the ascii encoding used by default during serialization, if you want to output the real Chinese, you need to specify ensure_ascii=False: for more in-depth analysis, it should be that the dJSON object is not a simple unicode implementation, but contains a mixed unicode encoding and a string that has been encoded with utf-8.
Related issue number (if applicable)
Checks
Before fixed:
After:
- [x] I've run
format.shto lint the changes in this PR. - [x] I've included any doc changes needed.
- [x] I've made sure the relevant tests are passing (if applicable).
actually @DSYZayn could you double check all the places we dump conversation data to json files are fixed?
@infwinston Not yet, I only modified the three files that we needed urgently. I'm glad to fix the remain ones, there are many files wait to fix.
@infwinston I've fixed all the potential error when dumps data to json file, and I found that httpx.AsyncClient usually cause error when user's network is not good, so I add an environment argument to fix it in openai_api_server.py. But still it will cause the same question when user run gradio_web_server or gradio_web_server_multi, but that's problem of upstream gradio. I suggest user mannually modified in this line. Exact path to the config file is: ~/path-to-venv/lib/python3.9/site-packages/httpx/_config.py ==> line #369
@infwinston Excuse me, I noticed it has been double review, took a long time and the bot got a successful check too. Maybe we can think about the possibility of merging it in the main branch😊. As you can see, all changes are basically safe and no aggressiveness.