kernel-memory
kernel-memory copied to clipboard
Make ITextEmbeddingGenerator.CountTokens and ITextGenerator.CountTokens ValueTask<int>
Right now these are synchronous, but if you're using an online service to implement these (i.e. LLama.cpp server) then these need to be able to return async responses. Having it return ValueTask would be greatly helpful.
Inversely GenerateEmbeddingAsync could be ValueTask as well.