bennmann
bennmann
Performance is ok, 5 tokens per second output for 6900Xt with the largest RWKV 14B model - Nvidia can do better with their tensor cores but 30X0 has ram limitations...
Update August 2023: I kind of dislike containers and usually prefer pure metal, but the below method should work (untested): Install AMD drivers (may need to chown _apt amd_driver.deb to...
cbe00741b0732cfe9c7b27924811903bd566cb98
did a git pull, then similar issues arise: ``` /LocalAI$ http://localhost:8080/v1/chat/chat/completions -H "Content-Type: application/json" -d '{ "model": "rwkv.cpp-1.5b-11x.bin", "messages": [{"role": "user", "content": "Say this is a test!"}], "temperature": 0.7 }'...
i ran out of disk space, and having a time getting my environment stable again.... i will report back in some time
ok, hopes this helps someone else too - containers kept stacking up in docker and ate my space away: had to `~$ docker system prune -a -f ` then i...
> today i tested this way, and confirmed all the folders structures were chown'd to my user i tried the local make build install and also get an RWKV HTTP...
Has anyone gotten past this http 500 issue?
Yes to the token and same issue
looks like some filename variables may not be gracefully accounting for the example/rwkv folder structure now? this is closer though (using the docker-compose method) ``` $ docker-compose up -d --build...