transformers.js icon indicating copy to clipboard operation
transformers.js copied to clipboard

Support nomic-ai/nomic-embed-vision-v1.5

Open markthethomas opened this issue 7 months ago • 1 comments

Model description

hey there! Was looking to use nomic-ai/nomic-embed-vision-v1.5 since I'm using the text version so I could support image / text queries using the same semantic space, but getting this error when I tried:


    at OnnxruntimeSessionHandler (/Users/....redacted..../node_modules/onnxruntime-node/lib/backend.ts:16:30)
    at <anonymous> (/Users/....redacted....node_modules/onnxruntime-node/lib/backend.ts:61:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)```

### Prerequisites

- [X] The model is supported in Transformers (i.e., listed [here](https://huggingface.co/docs/transformers/index#supported-models-and-frameworks))
- [X] The model can be exported to ONNX with Optimum (i.e., listed [here](https://huggingface.co/docs/optimum/main/en/exporters/onnx/overview))

### Additional information

_No response_

### Your contribution

I took a swing at re-converting to onnx using the provided script, but ran into some errors. It seems like there's already an `onnx` directory in the model files area, too, so that may be unnecessary

markthethomas avatar Jul 14 '24 15:07 markthethomas