solomonmanuelraj
solomonmanuelraj
Screenshot attached
Hi team, thanks for your quick response. I want to qunatize the model with 8 bit. in this case load_in_8bit=False will give the 8 bit quantized model ? this option...
Thanks a lot for your quick response. it was very useful. yes. i need to verify the accuracy. the quantized model will be poor in accuracy. Problem statement : my...
@fxmarty when i convert the owl-vit model into onnx file ( using optimum-cli export onnx -m google/owlvit-base-patch16 --task 'zero-shot-object-detection' --framework 'pt' owlvit_onnx) default opset is 14 it is taking (...
Thanks for the update. Quantization simulation mimic the accuracy of a quantized target. It is good. Whether it will mimic the inference time and memory size, MAC of a quantized...
@SlongLiu @tdzdog same problem ( like: 'the cat sit on the table. the dog on the ground.' - use sentences for multiple classes prompt) Is this problem solved? thanks.
Hi Team, Need your help to convert owl-vit model (OwlViTForObjectDetection) into onnx file. ########################################################################################## from PIL import Image from transformers import OwlViTProcessor, OwlViTForObjectDetection model_id = "google/owlvit-base-patch16" owlbit8_model = OwlViTForObjectDetection.from_pretrained(model_id,device_map="auto",load_in_8bit=True) owlbit8_model.save_pretrained("local...
thanks for your quick response.
@fxmarty , thanks for your quick response. yes i used your above code for the dynamic quantization on the owl-vit model. when i refer (https://www.philschmid.de/optimizing-transformers-with-optimum - 5. Test inference with...