Alexander Wu

@kevb10 @javadan @rmulligan @samuelmukoti thanks all for your nice comments and point out the problem. @iorisa could you plz check this?

Please resolve all conflicts and Review comments

Frankly, I think this implementation is probably better than the current rsp_cache. It doesn't pollute our commit history, and it's only minimal changes (it seems)

试试最新的版本？

@joschkabraun That really isn’t enough. Our existing cache takes over almost all network requests. Maybe it would be better at http layer

你好。提交的程序请遵守PEP8

This problem actually comes from the poor Instruction Following of gpt-3.5-turbo. gpt-4 basically does not have this problem

需要解决conflicts

we've RPM setting so far, but no TPM setting

Streaming not supported?