Fix BetterTransformer compatibility with transformers >= 4.49

[AimoreRRD]

Summary

• Handle RuntimeError when importing BetterTransformer with transformers >= 4.49
• Gracefully disable BetterTransformer optimization when unavailable due to version incompatibility
• Maintain backward compatibility with older transformers versions

Problem

BetterTransformer from optimum v1.27.0 is deprecated and incompatible with transformers >= 4.49, causing a RuntimeError that prevents infinity_emb from starting.

Solution

• Wrap BetterTransformer imports in try-except blocks
• Add BETTERTRANSFORMER_AVAILABLE flag to track availability
• Skip BetterTransformer optimization when not available, logging info message
• Continue normal operation without optimization

Test plan

• Test with transformers 4.53.3 and optimum 1.27.0
• Verify infinity_emb starts successfully with Qwen3 models
• Confirm graceful fallback when BetterTransformer unavailable
• Check that existing functionality remains unchanged

[AimoreRRD]

Fixes #642

[greptile-apps]

Greptile Summary

This PR fixes a critical compatibility issue between BetterTransformer and newer versions of the transformers library. The main problem was that BetterTransformer from optimum v1.27.0 throws a RuntimeError when used with transformers >= 4.49, causing infinity_emb to fail at startup.

The solution implements graceful degradation by wrapping BetterTransformer imports in try-catch blocks within acceleration.py. A new BETTERTRANSFORMER_AVAILABLE flag tracks whether BetterTransformer can be successfully imported. When unavailable due to version incompatibilities, the system logs an informational message and continues operation without the optimization rather than crashing.

The changes modify two key functions:

• check_if_bettertransformer_possible() now returns False early if BetterTransformer is unavailable
• to_bettertransformer() skips optimization and returns the original model when BetterTransformer is unavailable

The pyproject.toml update bumps the test transformers dependency to 4.53.3, ensuring the test suite validates the fix under the problematic version conditions. This approach maintains backward compatibility - users with older library versions can still benefit from BetterTransformer optimizations, while users with newer versions get a stable service without the optimization.

This fits into the codebase's broader acceleration framework, which conditionally applies various optimizations based on available libraries and configurations. The change follows the existing pattern of defensive checks (like device compatibility and environment variable checks) that already exist in the acceleration module.

Confidence score: 4/5

• This PR addresses a real compatibility issue with a robust fallback mechanism
• The implementation follows defensive programming principles and maintains backward compatibility
• The acceleration.py file requires attention due to the new import handling logic and early return paths

_{2 files reviewed, 1 comment}

_{Edit Code Review Bot Settings | Greptile}