Damian Kalinowski comments

Results 26 comments of


                                            Damian Kalinowski

Ability to detect client disconnection

Thanks @zjd1988, it solved my problem. #2204 PR introduces getter for TcpConnection ptr which allows installing custom close callback per connection. Can we merge it? @zjd1988 @fantasy-peak

Ability to detect client disconnection

@zjd1988 @fantasy-peak It resolved my issue, however issue still persists when I don't move with the workload to separate thread. What do I mean by that: It **works**, when I...

installing the ovms binary deployment on Ubuntu 24.04

This is good idea, we will consider adding it

OpenAI API completions endpoint - Not working as expected

Hello @anandnandagiri You are trying to serve the the model directly, with no continuous batching pipeline. In such scenario the model is exposed for single inference via standard TFS/KServe APIs...

OVMS model loading time is much longer than openvino model loading time and crashes from time to time

The comparision is between using Optimum Intel (ov_load_model) vs GenAI (ovms). It is not comparision of the same pipeline types. Can you change optimum-intel model loading to loading the model...

OVMS model loading time is much longer than openvino model loading time and crashes from time to time

Do you monitor RAM/vRAM usage during model reload? The scenario you have showcased in the python script switches base_path of a model. OVMS order of operations is that it always...