Skip to content

sbert normalize_embeddings 报错 #1853

Closed
@CurtainRight

Description

@CurtainRight

报错信息
sbert向量进行归一化报错,不归一化不报错。

  • Ascend环境
    image

  • Software Environment / 软件环境 (Mandatory / 必填):
    image
    image

Expected behavior / 预期结果 (Mandatory / 必填)
image

Screenshots/ 日志 / 截图 (Mandatory / 必填)
[ERROR] KERNEL(82890,fffe8fe3f1e0,python3.9):2024-12-02-03:09:35.241.039 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_mod.cc:260] Launch] Kernel launch failed, msg: Acl compile and execute failed, op_type_:LpNorm


  • Ascend Error Message:

E29999: Inner Error!
E29999: 2024-12-02-03:09:35.236.796 [SubGraphOpt][Compile][ProcFailedCompTask] Thread[281460975792608] recompile single op[LpNorm2] failed[FUNC:ProcessAllFailedCompileTasks][FILE:tbe_op_store_adapter.cc][LINE:961][THREAD:85252]
TraceBack (most recent call last):
[SubGraphOpt][Compile][ParalCompOp] Thread[281460975792608] process fail task failed[FUNC:ParallelCompileOp][FILE:tbe_op_store_adapter.cc][LINE:1009][THREAD:85252]
[SubGraphOpt][Compile][CompOpOnly] CompileOp failed.[FUNC:CompileOpOnly][FILE:op_compiler.cc][LINE:1112][THREAD:85252]
[GraphOpt][FusedGraph][RunCompile] Failed to compile graph with compiler Normal mode Op Compiler[FUNC:SubGraphCompile][FILE:fe_graph_optimizer.cc][LINE:1420][THREAD:85252]
Call OptimizeFusedGraph failed, ret:-1, engine_name:AIcoreEngine, graph_name:partition0_rank1_new_sub_graph1[FUNC:OptimizeSubGraph][FILE:graph_optimize.cc][LINE:119][THREAD:85252]
subgraph 0 optimize failed[FUNC:OptimizeSubGraphWithMultiThreads][FILE:graph_manager.cc][LINE:1012][THREAD:84819]
build graph failed, graph id:1, ret:-1[FUNC:BuildModelWithGraphId][FILE:ge_generator.cc][LINE:1608][THREAD:84819]
[Build][SingleOpModel]call ge interface generator.BuildSingleOpModel failed. ge result = 4294967295[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161][THREAD:84819]
[Build][Op]Fail to build op model[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145][THREAD:84819]
build op model failed, result = 500002[FUNC:ReportInnerError][FILE:log_inner.cpp][LINE:145][THREAD:84819]

(Please search "CANN Common Error Analysis" at https://www.mindspore.cn for error code description)


  • C++ Call Stack: (For framework developers)

mindspore/ccsrc/transform/acl_ir/acl_utils.cc:379 Run

[ERROR] DEVICE(82890,fffe8fe3f1e0,python3.9):2024-12-02-03:09:35.241.072 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:1156] LaunchKernel] Launch kernel failed, kernel full name: Default/LpNorm-op0

RuntimeError Traceback (most recent call last)
Cell In[5], line 3
1 # 向量预测
2 text = ['他开车去了体育场。']
----> 3 print(model_emb.encode(text, normalize_embeddings=True))

File /home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/sentence/sentence_transformer.py:236, in SentenceTransformer.encode(self, sentences, prompt_name, prompt, batch_size, show_progress_bar, output_value, precision, convert_to_numpy, convert_to_tensor, normalize_embeddings)
234 all_embeddings = np.asarray([emb.float().asnumpy() for emb in all_embeddings])
235 else:
--> 236 all_embeddings = np.asarray([emb.asnumpy() for emb in all_embeddings])
237 elif isinstance(all_embeddings, np.ndarray):
238 all_embeddings = [ops.from_numpy(embedding) for embedding in all_embeddings]

File /home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindnlp/sentence/sentence_transformer.py:236, in (.0)
234 all_embeddings = np.asarray([emb.float().asnumpy() for emb in all_embeddings])
235 else:
--> 236 all_embeddings = np.asarray([emb.asnumpy() for emb in all_embeddings])
237 elif isinstance(all_embeddings, np.ndarray):
238 all_embeddings = [ops.from_numpy(embedding) for embedding in all_embeddings]

File /home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/common/_stub_tensor.py:49, in _stub_method..fun(*arg, **kwargs)
47 stub = arg[0]
48 arg = (stub.stub_sync(),) + arg[1:]
---> 49 return method(*arg, **kwargs)

File /home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/mindspore/common/tensor.py:1055, in Tensor.asnumpy(self)
1053 if self.has_init:
1054 self.init_data()
-> 1055 return Tensor_.asnumpy(self)

RuntimeError: Launch kernel failed, name:Default/LpNorm-op0


  • C++ Call Stack: (For framework developers)

mindspore/ccsrc/runtime/pynative/op_runner.cc:624 LaunchKernels

Additional context / 备注 (Optional / 选填)
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions