infinity
infinity copied to clipboard
Fix BetterTransformer compatibility with transformers >= 4.49
Summary
• Handle RuntimeError when importing BetterTransformer with transformers >= 4.49 • Gracefully disable BetterTransformer optimization when unavailable due to version incompatibility • Maintain backward compatibility with older transformers versions
Problem
BetterTransformer from optimum v1.27.0 is deprecated and incompatible with transformers >= 4.49, causing a RuntimeError that prevents infinity_emb from starting.
Solution
- Wrap BetterTransformer imports in try-except blocks
- Add
BETTERTRANSFORMER_AVAILABLEflag to track availability - Skip BetterTransformer optimization when not available, logging info message
- Continue normal operation without optimization
Test plan
- [x] Test with transformers 4.53.3 and optimum 1.27.0
- [x] Verify infinity_emb starts successfully with Qwen3 models
- [x] Confirm graceful fallback when BetterTransformer unavailable
- [x] Check that existing functionality remains unchanged
Fixes #642