关于多张GPU微调大模型的问题

[yanhan111]

请问如果我想使用旧版本的“internlm-xcomposer-7b”进行两张3090进行微调，应该如何修改代码。我发现最新的多卡运行代码无法适用旧版本的模型

[yuhangzang]

Please check the code here

[yanhan111]

我使用了accelerate进行两张3090的demo，但是仍然报错File "/home/ubuntu/data/syh/C4MMD-main/C4MMDmain/CoT_module.py", line 210, in
response1 = model.generate(**inputs)
File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/internlm-xcomposer-7b/modeling_InternLM_XComposer.py", line 204, in generate
out_embeds = self.internlm_model.generate(inputs_embeds=prompt_embeds,
File "/home/ubuntu/anaconda3/envs/C4MMD/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/ubuntu/anaconda3/envs/C4MMD/lib/python3.8/site-packages/transformers/generation/utils.py", line 1681, in generate
return self.beam_search(
File "/home/ubuntu/anaconda3/envs/C4MMD/lib/python3.8/site-packages/transformers/generation/utils.py", line 3091, in beam_search
model_kwargs["past_key_values"] = self._reorder_cache(model_kwargs["past_key_values"], beam_idx)
File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/internlm-xcomposer-7b/modeling_InternLM.py", line 1243, in _reorder_cache
reordered_past += (tuple(
File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/internlm-xcomposer-7b/modeling_InternLM.py", line 1244, in
past_state.index_select(0, beam_idx)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument index in method wrapper__index_select)

Process finished with exit code 1

[yuhangzang]

Can u provide more details about your training script?

For example, you may run the finetune_lora.sh and modify the GPUS_PER_NODE == 2.