web-rwkv icon indicating copy to clipboard operation
web-rwkv copied to clipboard

Expose a C api?

Open Yorizuka opened this issue 1 year ago • 4 comments

First off, I would like to just say that web-rwkv is really well made, Unlike most things touching AI (especially with an AMD card) it just works! I was testing via ai00_server.

I now want to use this in my own project, but its not in rust so, any plans on eventually exposing a C api to this library? This would allow people to use web-rwkv from other languages. (In my case its D)

Is a C api on the roadmap, if not would you accept one if one was written for you? (I don't know rust, but I could give it a try)

Yorizuka avatar Aug 13 '24 09:08 Yorizuka

Thanks! There is a C ffi exists (here). It's not as flexible but is simple to use and extend. Feel free to extend it for your own usage, or reach out if you need any specific APIs.

cryscan avatar Aug 17 '24 06:08 cryscan

The link https://github.com/cryscan/web-rwkv-ffi seems to be dead (404)

Yorizuka avatar Aug 17 '24 10:08 Yorizuka

Ah, I make it public now.

cryscan avatar Aug 17 '24 14:08 cryscan

The link https://github.com/cryscan/web-rwkv-ffi seems to be dead (404)

Is that helpful to your application?

cryscan avatar Aug 20 '24 07:08 cryscan

Thank you for making it public, sadly the api seems a little too limited, I was specifically hoping to take advantage of batching for high throughput and to use a custom sampler. It would be nice to get access the raw logits.

Yorizuka avatar Aug 21 '24 21:08 Yorizuka