infinity

infinity copied to clipboard

Maintainer: Breaking CI / Python installs

1

### System Info I am not sure why the CI keeps breaking. Currently, the downloaded packages are cached to Github / via GH actions. Example: https://github.com/michaelfeil/infinity/actions/runs/11286542174/job/31391069817 Observation: Maybe happens when...

michaelfeil

Abstraction for `resolve_torch_dtype_device(dtype: Dtype, device: Device) -> tuple[quantization_type, torch.device, torch.dtype]`

1

### Feature request Too much boilerplate template: Resolves loading, quantization, and device Eg. if device: auto -> torch.cuda.is_available() -> cuda or mps. dtype: float32 -> float32, no quantization dtype: float16...

michaelfeil

[Doc] Documentation on how to run infinity on AWS Inf2

14

### Feature request Hello, I would like to know if there are any kind of configuration I have to make to run infinity as a docker container inside an inf2...

marcomarinodev

cli to set max_seq_length = 2048

1

### Feature request embedding_model = os.getenv("EMBEDDING_MODEL_FOR_INTENT_TRAINING", "jinaai/jina-embeddings-v2-base-zh") I requested the model jinaai/jina-embeddings-v2-base-zh through **infinity**. Is there any method to set the model's max_seq_length = 2048? ### Motivation Is there any...

Liwan-Chen

WIP: add back better_transformer without version check

6

POC to see if bettertransformer can simply be used with transformers>4.49 Extracted code from optimum repo to here: https://github.com/wirthual/better_transformer Had to update some dependencies to make it work.

wirthual

Qwen3-Embedding models do not work

12

### Model description The qwen3 models easily outperform nearly every other open source model for embeddings, however it does not work in infinity due to outdated transformers. My docker compose...

GhostDog98

Bug/semaphore lock

5

## Related Issue ## Checklist - [x] I have read the [CONTRIBUTING](https://github.com/michaelfeil/infinity/tree/main?tab=readme-ov-file#contribute-and-develop) guidelines. - [ ] I have added tests to cover my changes. - [x] I have updated the...

vasilypht

Running infinity cli with click >8.2.1 fails with TypeError: Secondary flag is not valid for non-boolean flag

3

### System Info As seen in #649 Workaround: Pin click to

wirthual

WIP: add changes for 3.13 support

2

Related to #594 ## Related Issue ## Checklist - [ ] I have read the [CONTRIBUTING](https://github.com/michaelfeil/infinity/tree/main?tab=readme-ov-file#contribute-and-develop) guidelines. - [ ] I have added tests to cover my changes. - [...

wirthual

Optimum 2.0.0: Bettertransformer deprecated

5

### System Info When install with pip install infinity[all] it will install optimum 2.0.0. Resulting in the following error when using the infinity cli. ``` infinity_emb v2 --help Traceback (most...

noobHappylife