RFC: LLM -> StrInStrOutLLM

[dev2049]

Find this interface name very confusing (not specific enough). Thoughts on updating? Open to other names, just want it to be specific.

Think it's backwards compat but very well might be overlooking something

[dev2049]

what's opposite of "batched"? non-batched? could also call it something like that (NonBatchedLLM)

[vowelparrot]

Hm I agree the _call naming is confusing and it'd be easier if we went the other way (generate -> batch_generate). Could call it something like _call_single() or something

[dev2049]

Hm I agree the _call naming is confusing and it'd be easier if we went the other way (generate -> batch_generate). Could call it something like _call_single() or something

i like idea of this being generate and other one being batch_generate, just worried that that may be too big of a change at this pt

i like generate more than call, call doesn't mean much to me. i also think its more than just "single", which to me would imply you're returning a single Generate object. but this is even less than that, its a single string output

[dev2049]

another idea: SimpleLLM