Skip to content

IPEXModel is no longer support torchscript models #78

@zhanxw

Description

@zhanxw

When I tried this notebook (https://github.com/IntelLabs/fastRAG/blob/main/examples/optimized-embeddings.ipynb), I got this error:

>>>
>>> doc_embedder.warm_up(); query_embedder.warm_up()

config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 747/747 [00:00<00:00, 130kB/s]
Loading IPEX ST Transformer model
Traceback (most recent call last):
  File "", line 1, in 
  File "/endosome/archive/PCDC/PCDC_Core/xzhan9/mycode/llama.panda/fastrag/fastRAG/fastrag/embedders/ipex_embedder.py", line 153, in ipex_model_warm_up
    self.embedding_backend = _IPEXSentenceTransformersEmbeddingBackend(
  File "/endosome/archive/PCDC/PCDC_Core/xzhan9/mycode/llama.panda/fastrag/fastRAG/fastrag/embedders/ipex_embedder.py", line 136, in __init__
    self.model = _IPEXSentenceTransformer(
  File "/home2/xzhan9/software/miniconda3/envs/fastrag/lib/python3.8/site-packages/sentence_transformers/SentenceTransformer.py", line 318, in __init__
    modules = self._load_auto_model(
  File "/endosome/archive/PCDC/PCDC_Core/xzhan9/mycode/llama.panda/fastrag/fastRAG/fastrag/embedders/ipex_embedder.py", line 120, in _load_auto_model
    transformer_model = _IPEXSTTransformers(
  File "/home2/xzhan9/software/miniconda3/envs/fastrag/lib/python3.8/site-packages/sentence_transformers/models/Transformer.py", line 76, in __init__
    self._load_model(model_name_or_path, config, cache_dir, backend, **model_args)
  File "/endosome/archive/PCDC/PCDC_Core/xzhan9/mycode/llama.panda/fastrag/fastRAG/fastrag/embedders/ipex_embedder.py", line 40, in _load_model
    self.auto_model = IPEXModel.from_pretrained(
  File "/home2/xzhan9/software/miniconda3/envs/fastrag/lib/python3.8/site-packages/optimum/modeling_base.py", line 438, in from_pretrained
    return from_pretrained_method(
  File "/home2/xzhan9/software/miniconda3/envs/fastrag/lib/python3.8/site-packages/optimum/intel/ipex/modeling_base.py", line 149, in _from_pretrained
    raise ValueError("IPEXModel is no longer support torchscript models.")
ValueError: IPEXModel is no longer support torchscript models.
>>>
>>> import torch
>>> torch.__version__
'2.4.1+cu121'

I am using Python 3.8.20 | packaged by conda-forge | (default, Sep 30 2024, 17:52:49).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions