clearml-serving
clearml-serving copied to clipboard
allegroai
ClearML - Model-Serving Orchestration and Repository Solution
Add the option to send exit-on-error parameter to tritonserver. Setting this to false prevents single models breaking a whole tritonserver instance. Fixes allegroai/clearml-serving#60
Hi I built clearml server by self-hosted I have followed [official tutorial](https://clear.ml/docs/latest/docs/clearml_serving/clearml_serving_setup) My config: ```bash CLEARML_WEB_HOST="http://localhost:8080" CLEARML_API_HOST="http://localhost:8008" CLEARML_FILES_HOST="http://localhost:8081" CLEARML_API_ACCESS_KEY="J1WS8MDJMI9MMMA9EMH6" CLEARML_API_SECRET_KEY="5eEYYHq3VMgf9DmQjfLtCiBBV09TgxImQmz8WDckKB0h6t3CIE" CLEARML_SERVING_TASK_ID="2f5e0563914e41b9a1334f841c39852b" ``` because CLEARML_WEB_HOST on port 8080, I have to...
We have setup clearml serving on Kubernetes including triton support. Our triton instance has no GPU, so deploying a model leads to the following error in the triton instance: `E0718...
### Describe the bug a clear and concise description of what the bug is. Hi there, it seems that when adding a pytorch model to the self-hosted clearml-serving the platform...
Restart `clearml-serving-inference` on `torch.cuda.OutOfMemoryError: CUDA out of memory.` helps inference container to clear GPU memory. It would be useful for LLM inference on `clearml-serving-inference` container (requires https://github.com/allegroai/clearml-helm-charts/blob/main/charts/clearml-serving/templates/clearml-serving-inference-deployment.yaml#L74 set to 1)...
Hello. I have some helper functions that are shared acros the `preprocess.py` files, so I'd like to refactor them. However, I'm not sure where I can put them, and how...
Hi I setup step by step from [official tutorial](https://github.com/allegroai/clearml-serving?tab=readme-ov-file#point_right-toy-model-scikit-learn-deployment-example). However, I see some error like this ``` Retrying (Retry(total=239, connect=239, read=240, redirect=240, status=240)) after connection broken by 'NewConnectionError(': Failed to...
Hello! I use ClearML free (the one without configuration vault stuff) + clearml-serving module When I spinned _docker-compose_ and tried to pull model from our s3, I've got an error...
From the docs, i can see that there are commands to add model to an endpoint, and also add a model monitoring via the auto-update command. I can't seem to...
Hello, I see TorchServe engine support mentioned in the Readme but cannot find any way to actually use it. Is it available?
