Search results

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:7 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

Status: Open.

#105 In OpenGVLab/OmniQuant;

· forcekkk opened on Apr 11, 2025

I encounter a error: "AttributeError: 'LlamaAttention' object has no attribute 'rotary_emb'",when i run code with llama-1-7b. It happened in int_llama_layer.py: self.rotary_emb = copy.deepcopy(org_module.rotary_emb)

Status: Open.

#104 In OpenGVLab/OmniQuant;

· WX-yh opened on Jan 5, 2025

I encounter a error: "AttributeError: 'LlamaAttention' object has no attribute 'rotary_emb'",when i run code with llama-1-7b.

Status: Open.

#103 In OpenGVLab/OmniQuant;

· WX-yh opened on Jan 5, 2025

Fail to reproduce the result of w2a16 using llama2 7b

Status: Open.

#102 In OpenGVLab/OmniQuant;

· stackByStack opened on Dec 16, 2024

Obtained different PPL for Wikitext and C4 compared to results reported in the paper

Status: Open.

#95 In OpenGVLab/OmniQuant;

· yc2367 opened on Sep 12, 2024

Error when evaluating MMLU

Status: Open.

#91 In OpenGVLab/OmniQuant;

· zjq0455 opened on Sep 4, 2024

how to enable llama3-8b int4 awq models

Status: Open.

#90 In OpenGVLab/OmniQuant;

· FlexLaughing opened on Aug 29, 2024

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat2 in method wrapper_CUDA_bmm)

Status: Open.

#89 In OpenGVLab/OmniQuant;

· mcpaulgeorge opened on Aug 5, 2024

The version of transformers, auto_gptq, autoawq

Status: Open.

#88 In OpenGVLab/OmniQuant;

· zhangfzR opened on Aug 5, 2024

Quantify tinyllama-1.1B-Chat-v1.0, a CUDA assertion error occurred

Status: Open.

#87 In OpenGVLab/OmniQuant;

· causeof-you opened on Jul 12, 2024

[New Feature] Seek MLA Supported by Smooth

Status: Open.

#86 In OpenGVLab/OmniQuant;

· RanchiZhao opened on Jul 10, 2024

question about let

Status: Open.

#85 In OpenGVLab/OmniQuant;

· mxjmtxrm opened on Jul 8, 2024