clearml-serving icon indicating copy to clipboard operation
clearml-serving copied to clipboard

Published 20 hours ago • verified publisher icon allegroai

Cuda oom handler [continuation]

Open IlyaMescheryakov1402 opened this issue 7 months ago • 0 comments

In https://github.com/allegroai/clearml-serving/pull/75 was mentioned that CLEARML_SERVING_NUM_PROCESS has to be 1 (for k8s instance it can be set in https://github.com/allegroai/clearml-helm-charts/blob/main/charts/clearml-serving/templates/clearml-serving-inference-deployment.yaml#L77C21-L77C48), this PR set the same variable for docker-compose instance.

It also fixes requirements - add torch for attempts to clear GPU memory (it can be changed if served models require other version of torch)

IlyaMescheryakov1402 avatar Jul 25 '24 10:07 IlyaMescheryakov1402