infinity

infinity copied to clipboard

ValueError: No onnx files found

4

Hello, First of all thank you very much for this tool! I am trying it out (on CPU) with the following code: ```python DEVICE = os.environ.get("DEVICE", "cpu") MODEL_NAME = 'sentence-transformers/all-MiniLM-L6-v2'...

netw0rkf10w

Add revision and trust_remote_code to from_pretrained calls

1

I was trying out `Snowflake/snowflake-arctic-embed-m-long` with onnx engine ``` infinity_emb --model-name-or-path Snowflake/snowflake-arctic-embed-m-long --revision 08e7a4449e3f07709fb9387bc3172d393a6cc5e2 --engine optimum --device cpu --dtype float32 --batch-size 4 --port 8000 --trust-remote-code ``` But trust_remote_code is not...

chiragjn

Scores slightly off/get rounded up to 1.0

4

### System Info Hey I am working on a custom component in Haystack right now to build an Infinity Reranker. Might turn it into an Haystack Integration if I get...

ruben-vb

Tensor-parallelism for multi-gpu support

3

### Feature request Being able to split models into multiple GPUs, as with vllm/aphrodite engine for LLMs. ### Motivation It would be extremely helpful to be able to split larger...

SalomonKisters

Different results with mixedbread-ai/mxbai-embed-large-v1 model

1

### System Info The full command line used that causes issues: `docker run --rm -p 7997:7997 michaelf34/infinity:latest --model-name-or-path mixedbread-ai/mxbai-embed-large-v1 --port 7997` OS version: macos Model being used: mixedbread-ai/mxbai-embed-large-v1 Hardware used...

stephenleo

API Key Authentication for Michaelfeil Infinity

1

### Model description Could you please provide guidance on how to enable API_KEY authentication? Alternatively, is there a plan to implement API_KEY authentication similar to OpenAI's approach? ### Open source...

AjayKarma05

Support for instructur/instructor-xl models

9

Can you please support the instructor models here? https://github.com/xlang-ai/instructor-embedding These are arguably the best models for their sizes.

BBC-Esq

Refactor: async usage in BatchHandler

1

michaelfeil

Fixing the astop option

michaelfeil

Error in offline mode with `trust_remote code`: SFR-Embedding-Mistral and nomic does not work without `einops`

4

### Model description You have mentioned that sfr-embedding model is supported along with all other huggingface embedding models (ref.nomic). However, both are not working : infinity | ERROR 2024-03-21 14:35:59,554...

prasannakrish97