Miłosz Żeglarski
Miłosz Żeglarski
Thank you for bringing this to our attention. It seems that OpenCV installation with APT makes the image 1.8 GB large. We'll try to make it smaller and get back...
The PR with shrinking the image - https://github.com/openvinotoolkit/model_server/pull/1459
Hi, Did you have a chance to take a look at this doc: https://github.com/openvinotoolkit/model_server/blob/releases/2022/1/docs/python-benchmarking-client-16feb.pdf ? @krzyczar is the author of the client and above document, so hopefully he can clarify...
The log is a misleading. It's "auto" parameter that is forbidden, but dynamic shapes (with -1 dimensions) should work. Could you remove `"shape": "auto"` from your config and try again?
I don't think it's something to fix on the client side. My guess would be networking especially that you say it always works when you deploy locally and the issue...
Could you check your model with OpenVINO benchmark app: https://docs.openvino.ai/2023.3/openvino_sample_benchmark_tool.html ? Run with `-d GPU` option to run on GPU. Please also share the command you use to start OVMS.
Could you pull latest changes and try again?
I suppose earlier we limited results in `generate()` method. With that change should we remove it from `generate()`?
Seems like new limit on OV version has been added to optimum-intel: https://github.com/huggingface/optimum-intel/commit/cff990218b93313519b15f15bf1005a90ca8aeab I'll try to find out why it was added and when we could lift that limitation. Meanwhile...
Duplicate of: https://github.com/openvinotoolkit/openvino.genai/pull/953