Kirat Pandya comments

Repositories
Issues
Comments

Results 12 comments of


                                            Kirat Pandya

Investigate PagedAttention KV-cache memory management for faster inference

#2813 only covers "same prompt, multiple output", not "multiple prompt, multiple output".

Docker Issus ''Illegal instruction''

> On a project with a million dependencies an libraries this might be a problem, but as there is no dependencies and builds on anything and thus compilation shouldn't pose...