Qwen3 not supported for wrong transformer version #32
Description
Hello, I was trying to use Qwen/Qwen3-Embedding-4B as embedder model but I got an error version with the transformers library (the required version is this transformers>=4.51.0). I tried to create a custom docker container with an updated version but I got error with BetterTransformers
RuntimeError: BetterTransformer requires transformers<4.49 but found 4.53.3. `optimum.bettertransformer` is deprecated and will be removed in optimum v2.0.\n
mkhqi4md01rrhl
[info]
raise RuntimeError(\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/optimum/bettertransformer/__init__.py", line 20, in <module>\n
mkhqi4md01rrhl
[info]
from optimum.bettertransformer import ( # type: ignore[import-untyped]\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/infinity_emb/transformer/acceleration.py", line 11, in <module>\n
mkhqi4md01rrhl
[info]
from infinity_emb.transformer.acceleration import (\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/infinity_emb/transformer/classifier/torch.py", line 8, in <module>\n
mkhqi4md01rrhl
[info]
from infinity_emb.transformer.classifier.torch import SentenceClassifier\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/infinity_emb/transformer/utils.py", line 9, in <module>\n
mkhqi4md01rrhl
[info]
from infinity_emb.transformer.utils import get_lengths_with_tokenize\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/infinity_emb/inference/batch_handler.py", line 39, in <module>\n
mkhqi4md01rrhl
[info]
from infinity_emb.inference.batch_handler import BatchHandler\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/infinity_emb/inference/__init__.py", line 4, in <module>\n
mkhqi4md01rrhl
[info]
from infinity_emb.inference import (\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/infinity_emb/engine.py", line 11, in <module>\n
mkhqi4md01rrhl
[info]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
mkhqi4md01rrhl
[info]
from infinity_emb.engine import AsyncEmbeddingEngine, AsyncEngineArray # noqa: E402\n
mkhqi4md01rrhl
[info]
File "/usr/local/lib/python3.11/dist-packages/infinity_emb/__init__.py", line 27, in <module>\n
mkhqi4md01rrhl
[info]
from infinity_emb.engine import AsyncEngineArray, EngineArgs\n
mkhqi4md01rrhl
[info]
File "/embedding_service.py", line 2, in <module>\n
mkhqi4md01rrhl
[info]
from embedding_service import EmbeddingService\n
mkhqi4md01rrhl
[info]
File "/handler.py", line 4, in <module>\n
mkhqi4md01rrhl
[info]
Traceback (most recent call last):\n
Is there any way to get this model running?