rwkv.cpp icon indicating copy to clipboard operation
rwkv.cpp copied to clipboard

Published 20 hours ago • verified publisher icon RWKV

Lanchain.js integration

Open ansarizafar opened this issue 2 years ago • 4 comments

Is it some how possible to use rwkv.cpp with langchain.js https://js.langchain.com/docs/

ansarizafar avatar Apr 12 '23 06:04 ansarizafar

If RWKV server from #17 is merged, it should be possible

CC @abetlen

saharNooby avatar Apr 12 '23 09:04 saharNooby

I don't want to use python or python based server. I want to use rwkv.cpp with langchain javascript version.

ansarizafar avatar Apr 12 '23 11:04 ansarizafar

@ansarizafar for server-side usage that would require runtime specific bindings (node-ffi / Deno FFI). One challenge is that you need to build the shared library on the processor you're targetting to get the best performance. For the python package I'm using scikit-build which allows you to build a nativ C/C++ project at install time.

abetlen avatar Apr 12 '23 14:04 abetlen

@abetlen I want to use rwkv model with langchain.js and bunjs https://bun.sh/. We can call native C/C++ libraries with Bun FFI module https://bun.sh/docs/api/ffi

ansarizafar avatar Apr 12 '23 15:04 ansarizafar