optimum-intel icon indicating copy to clipboard operation
optimum-intel copied to clipboard

Fail to execute quantized model on Ray

Open sherrytonger opened this issue 1 year ago • 0 comments

when execute the following code:

quantized_model = IPEXModel.from_pretrained(self.model_path).eval()
model_ref = ray.put(quantized_model )

Error message:

RuntimeError: Tried to serialize object __torch__.intel_extension_for_pytorch.quantization._quantize_utils.___torch_mangle_972.QuantizationDispatchModule which does not have a __getstate__ method defined!

Is there any workaround to this problem?

sherrytonger avatar Jul 22 '24 08:07 sherrytonger