infinity
infinity copied to clipboard
michaelfeil
Optimum 2.0.0: Bettertransformer deprecated
System Info
When install with pip install infinity[all] it will install optimum 2.0.0. Resulting in the following error when using the infinity cli.
infinity_emb v2 --help
Traceback (most recent call last):
File "/media/data4/yh/uvenv/infinity077/bin/infinity_emb", line 4, in <module>
from infinity_emb.cli import cli
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/__init__.py", line 7, in <module>
from infinity_emb.engine import AsyncEmbeddingEngine, AsyncEngineArray # noqa: E402
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/engine.py", line 11, in <module>
from infinity_emb.inference import (
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/inference/__init__.py", line 4, in <module>
from infinity_emb.inference.batch_handler import BatchHandler
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/inference/batch_handler.py", line 39, in <module>
from infinity_emb.transformer.utils import get_lengths_with_tokenize
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/transformer/utils.py", line 9, in <module>
from infinity_emb.transformer.classifier.torch import SentenceClassifier
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/transformer/classifier/torch.py", line 8, in <module>
from infinity_emb.transformer.acceleration import (
File "/media/data4/yh/uvenv/infinity077/lib/python3.12/site-packages/infinity_emb/transformer/acceleration.py", line 11, in <module>
from optimum.bettertransformer import ( # type: ignore[import-untyped]
ModuleNotFoundError: No module named 'optimum.bettertransformer'
Optimum installed:
pip freeze | grep optimum
optimum==2.0.0
Checking the optimum released notes, https://github.com/huggingface/optimum/releases/tag/v2.0.0 bettertransformer has been deprecated and removed.
Information
- [ ] Docker + cli
- [x] pip + cli
- [ ] pip + usage of Python interface
Tasks
- [ ] An officially supported CLI command
- [ ] My own modifications
Reproduction
Installed with
pip install infinity-emb[all]
Failed to run cli as mentioned above.
There is an experimental PR #641 which should work around this issue.
Can you try it out and report back if it works for you?
git clone -b better_transformer_clone https://github.com/michaelfeil/infinity.git
cd infinity/libs/infinity_emb/
pip install -e ".[all]"
I tried your branch and see a different error.
infinity_emb v2 --help
/home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/torch/onnx/_internal/registration.py:159: OnnxExporterWarning: Symbolic function 'aten::scaled_dot_product_attention' already registered for opset 14. Replacing the existing function with new function. This is unexpected. Please report it on https://github.com/pytorch/pytorch/issues.
warnings.warn(
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ /home/akk/micromamba/envs/infinity077/bin/infinity_emb:7 in <module> │
│ │
│ 4 if __name__ == '__main__': ╭──────────── locals ─────────────╮ │
│ 5 │ if sys.argv[0].endswith('.exe'): │ sys = <module 'sys' (built-in)> │ │
│ 6 │ │ sys.argv[0] = sys.argv[0][:-4] ╰─────────────────────────────────╯ │
│ ❱ 7 │ sys.exit(cli()) │
│ 8 │
│ │
│ /media/data4/yh/git/infinity/libs/infinity_emb/infinity_emb/cli.py:413 in cli │
│ │
│ 410 │ │ │ ) │
│ 411 │ │ │ │
│ 412 │ │ │ sys.argv.insert(1, "v2") │
│ ❱ 413 │ tp() │
│ 414 │
│ 415 │
│ 416 if __name__ == "__main__": │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:338 in __call__ │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:321 in __call__ │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:360 in get_command │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:342 in get_group │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:496 in get_group_from_info │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:590 in get_command_from_info │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:566 in get_params_convertors_ctx_param_name_from_function │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/main.py:900 in get_click_param │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/typer/core.py:427 in __init__ │
│ │
│ /home/akk/micromamba/envs/infinity077/lib/python3.12/site-packages/click/core.py:2793 in __init__ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: Secondary flag is not valid for non-boolean flag.
Thanks for trying, can you try to downgrade click:
pip install click==8.1.8