dapo时在UserWarning: None of the inputs have requires_grad=True. Gradients will be None一直卡住，直至timeout

[evyy1234]

如图 ，训练一会就卡住了，重启了很多次还是如此，不知道是什么原因，请大佬帮忙忙看下：
shell:
CUDA_VISIBLE_DEVICES=0,1,2,3 NPROC_PER_NODE=3 swift rlhf --rlhf_type grpo --model /opt/db/Qwen2___5-32B-Instruct --train_type lora --use_vllm true --dataset /opt/db/grpo/test.jsonl --max_completion_length 2024 --num_train_epochs 1 --learning_rate 1e-6 --beta 0.01 --gradient_accumulation_steps 1 --eval_steps 50 --save_steps 50 --temperature 0.4 --output_dir output --warmup_ratio 0.05 --dataloader_num_workers 4 --dataset_num_proc 4 --num_generations 3 --log_completions true --external_plugins /opt/db/grpo/rew.py --per_device_train_batch_size 1 --per_device_eval_batch_size 1 --reward_funcs thinkformat tool_acc --reward_weights 1.0 2.0 --vllm_gpu_memory_utilization 0.98 --vllm_max_model_len 27000 --multi_turn_func tool_tip_trick_multi_turn --deepspeed zero3 --epsilon_high 0.28 --dynamic_sample true --overlong_filter true

[hjh0119]

Dynamic sampling may require a significant amount of time. You can use the py-spy command to check where the process is getting stuck.

By the way, in some cases, using async mode with vLLM 0.8 and DeepSpeed ZeRO-3 may cause the process to hang. To avoid this issue, you can either downgrade vLLM to version 0.7.3 or switch to external mode

[sevenqhuang]

我也遇到过这种情况，它卡住的时候用nvidia-smi查看其实gpu的利用率是0，你可以试试训练用2卡，推理用2卡，即CUDA_VISIBLE_DEVICES=0,1,2,3 NPROC_PER_NODE=2 swift ... --num_infer_workers 2