fxmarty
fxmarty
Hi @hakan458 @tomaarsen, could you try Optimum 1.16.2 that includes https://github.com/huggingface/optimum/pull/1641?
@BBerabi @lidingsnyk Thank you for the details & apology for the late reply, it should be fixed by https://github.com/huggingface/optimum/pull/1722 ```python import torch import transformers from optimum.onnxruntime import ORTModelForCausalLM my_checkpoint =...
Thank you, I agree it would be very nice. This library https://github.com/marella/ctransformers is close to what you are suggesting, except that it is using GGML instead of ONNX Runtime for...
cc @SunMarc
@isaac-chung Apology for the convoluted error. Here is a patch for this one: https://github.com/isaac-chung/optimum/pull/2 There is then an other error about the naming of the model inputs. One approach you...
Yes, it appears there are no small open clip models on the Hub. Probably the most reasonable option (if you don't want to go into the hassle of addind a...
@isaac-chung Thank you. It appears that the `tokenize` function from open_clip and the `open_clip.tokenizer.HFTokenizer` do pad by default to `tokenizer.tokenizer.model_max_length`, while Transformers CLIP do not. It is unclear to me...
@isaac-chung Yes that is a good option as well! It is just that `overwrite_shape_and_generate_input` is not defined at the top level `OnnxConfig`, so you would need to define it there...
Hi @isaac-chung , could you check where is the argmax in the modeling code? What do you mean by `we only deal with fp16`?
Thank you! Can you check the dtype of `text` here https://github.com/mlfoundations/open_clip/blob/7b8dd2cbaf9cca13ca5b1defa6a321a145eb166c/src/open_clip/transformer.py#L558? A solution would be to use the ModelPatcher (see e.g. https://github.com/huggingface/optimum/blob/15b8d1eed4d83c5004d3b60f6b6f13744b358f01/optimum/exporters/onnx/model_patcher.py#L638-L651 & https://github.com/huggingface/optimum/blob/15b8d1eed4d83c5004d3b60f6b6f13744b358f01/optimum/exporters/onnx/model_configs.py#L1415-L1418) to cast the arg `text` in...