TypeChat icon indicating copy to clipboard operation
TypeChat copied to clipboard

feat: export createAxiosLanguageModel

Open abichinger opened this issue 2 years ago • 4 comments
trafficstars

Exporting createAxiosLanguageModel would solve the following problems

  1. support custom axios config (e.g. users can pass a proxy)
    related: #73 #55 #34
  2. ability to pass custom parameters to the endpoint /v1/chat/completions

abichinger avatar Aug 04 '23 14:08 abichinger

Personally I don't think the library should expose Axios at all, since it's an internal implementation detail.

For example, in another issue someone asked about how to take advantage of streaming responses from OpenAI. Best I can tell, Axios doesn't make that easy to do (or it might not provide the APIs to do so at all). I use this as a specific example, but my point is really that hooking up Axios or whatever language model provider API is easy enough because TypeChatLanguageModel just needs a single function at the moment.

Still, I'll let others weigh in here.

DanielRosenwasser avatar Aug 09 '23 22:08 DanielRosenwasser

@DanielRosenwasser thete are likely legit n/w level overrides desired, but middleware is likely the better way forward imo rather than anything axios/fetch/ky specific.

I agree with exposing the model the way it is with the single function 👍

snewell92 avatar Aug 11 '23 09:08 snewell92

I don't think TypeChat needs to provide proxy settings, as it can support proxy access through higher-level configurations, such as proxychains4.

Akimyou avatar Aug 14 '23 18:08 Akimyou

  1. I think the package shall use undici as this will be the default client in the future Node versions
  2. If anything needs to be exposed, then I think FetchEsque types from trpc can be borrowed, which can adapt to any fetch implementation.

moltar avatar Aug 15 '23 14:08 moltar