jacob-vincent-mink comments

Results 4 comments of


                                            jacob-vincent-mink

ONNX conversion of google/gemma-2b-it logits values are very different

Covering my bases... ``` import torch from optimum.onnxruntime import ORTModelForCausalLM model = ORTModelForCausalLM.from_pretrained('/path/to/gemma') logits = model(torch.LongTensor([[1,2,3,4,5]]), torch.Tensor([[1,1,1,1,1]]), torch.LongTensor([[0,1,2,3,4]]))['logits'] ``` The logits here look like the pure ORT version in the...

ONNX conversion of google/gemma-2b-it logits values are very different

@IlyasMoutawwakil thanks for the information! I will try this out and report back. It’s worth noting that the ONNX model I’m having trouble with was converted with the optimum-cli -...

ONNX conversion of google/gemma-2b-it logits values are very different

@fxmarty Thanks for your comments. All my experiments have been done on Windows. I did the following: 1. Create a new virtual environment 2. `pip install git+https://github.com/huggingface/transformers.git git+https://github.com/huggingface/optimum.git onnxruntime onnx...

ONNX conversion of google/gemma-2b-it logits values are very different

@fxmarty @nickrwann Looks like this is an optimum-cli on Windows issue. I ran it in WSL and got the following: ``` (venv2) jacob@W11JQ9MZZ2:/mnt/c/BEA/Work/SQUID/takoyaki/conversion$ optimum-cli export onnx --model google/gemma-2b-it --framework pt...