tiktoken
tiktoken copied to clipboard
Unable to route GET requests through proxy
I need to be able to call tiktoken in an environment where outbound traffic needs to be routed through a specific proxy. This is not possible with the current implementation of Tiktoken. One solution would be if read_file function in load.py would accept a session arg so the end user can pass a session with proxy.
same error, need some help!
Ideally, https://github.com/openai/tiktoken/blob/c0ba74c238d18b4824c25f3c27fc8698055b9a76/tiktoken/load.py#L143 should allow passing headers, proxies, verification, and other keyword arguments (kwargs) as documented in requests.
However, setting "offline" usage might be a better option for production use, but still requires an initial download.