Patrick Horn issues

Repositories
Issues
Comments

Results 3 issues of


                                            Patrick Horn

[BUG] Usability regression due to commit 0b34396

**Describe the bug** The commit in 0b34396 modified the license which makes redis unusable for my personal projects. This is a serious regression, and I would like to encourage that...

tritonserver HTTPServer does not detect EOF/connection close unless response is streaming

**Description** When a user performs a long-running inference request via HTTPServer, they may lose connection or intentionally abort the connection (ctrl-c from curl). Ideally, the HTTP server will detect this...

enhancement

Use first bad_words as extra parameters, and implement min-p

An approach for implementing #1154 The user-facing classes in BatchManager and `tensorrt_llm::executor::SamplingConfig` are not open source (the constructor implementation exists in .a files and will segfault if the class is...