请问下，无法支持驱动跳舞把，出来的效果很奇怪

[gangxu822]

video_0543_002_wan_vap.mp4

sex_7s_wan_vap.mp4

[yxbian23]

video_0543_002_wan_vap.mp4
sex_7s_wan_vap.mp4

Hi, thanks for trying our work! We think this may arise from some misuse and our data limitation:

• Possible Misuse: Because I saw there are strong distortion in your results, I guess you may use a very brief caption (too short to cover the specific dance semantic) for reference video and target video or a very complex reference video, where I encounter similar results in our early experiments.

• Our Data Limitation: As our report said, these semantic-controlled video data must be collected with a very high cost. So even we try to build the current largest dataset, we still find it is not enough to cover all the semantic in the daily life (empirically, we think it would need at least a million level dataset.)

  And unfortunately, the human motion pair data is the kind that are most difficult to collect (same motion, different skeleton, position), which only account for merely 1 % in our training data. So our model's generalizability is relatively limited compare to other semantics, such as concepts (E.g., change into a Labubu), stylizations (E.g., Ghibli), camera movements (E.g., basic translation, rotation, and Hitchcock camera movement), which can mostly perform well to a similar dance motion of our dataset. Below is a fresh example I just ran with a picture of my own to demonstrate the generalizability with the dance semantic in our dataset:

Reference Video:

woman-575.mp4

Reference Image:

Reference Caption:
A young woman with long brown hair and sunglasses stands in front of a colorful graffiti mural on a corrugated metal wall. She wears a form-fitting white tank top and light-wash ripped jeans. Initially, she brushes a hand through her hair before raising both hands to rest behind her head, elbows pointed outwards. She then begins to sway her hips and upper body from side to side in a steady, rhythmic dance. Her movement is fluid and confident, centered on her core. The camera remains completely static in a medium shot, keeping the woman centered in the frame. The background, marked by vertical shadow lines from the sun, is completely still, highlighting the subject's gentle, looping motion.

Target Caption:
A young man with short dark hair stands in front of a glowing city skyline at dusk, the sky fading into deep blue behind him. He wears a dark short-sleeve shirt with a pair of sunglasses hanging from the collar, along with a black backpack resting on both shoulders. At first, he stands relaxed, facing the camera with a calm expression. He then lifts both hands and places them loosely behind his head, elbows angled outward. As he holds this posture, he begins to gently sway his upper body from side to side in a slow, rhythmic motion, centered around his torso rather than his legs. The movement is smooth and unhurried, giving a sense of casual confidence. Throughout the sequence, the camera remains completely static in a medium shot, keeping the man clearly framed against the distant, sparkling city lights, which stay motionless and serene in the background.

Target Video:

wan_vap_demo.mp4

• Suggested Solution: We encourge to first try the semantic similar to our dataset. If you are interested in a specific domain and the domain semantic is a little far from our dataset, it would be better to have a little training to adapt (E.g., LoRA).

[gangxu822]

video_0543_002_wan_vap.mp4
sex_7s_wan_vap.mp4

Hi, thanks for trying our work! We think this may arise from some misuse and our data limitation:

• Possible Misuse: Because I saw there are strong distortion in your results, I guess you may use a very brief caption (too short to cover the specific dance semantic) for reference video and target video or a very complex reference video, where I encounter similar results in our early experiments.
• Our Data Limitation: As our report said, these semantic-controlled video data must be collected with a very high cost. So even we try to build the current largest dataset, we still find it is not enough to cover all the semantic in the daily life (empirically, we think it would need at least a million level dataset.)
  And unfortunately, the human motion pair data is the kind that are most difficult to collect (same motion, different skeleton, position), which only account for merely 1 % in our training data. So our model's generalizability is relatively limited compare to other semantics, such as concepts (E.g., change into a Labubu), stylizations (E.g., Ghibli), camera movements (E.g., basic translation, rotation, and Hitchcock camera movement), which can mostly perform well to a similar dance motion of our dataset. Below is a fresh example I just ran with a picture of my own to demonstrate the generalizability with the dance semantic in our dataset:

Reference Video:

woman-575.mp4
Reference Image:

Reference Caption: A young woman with long brown hair and sunglasses stands in front of a colorful graffiti mural on a corrugated metal wall. She wears a form-fitting white tank top and light-wash ripped jeans. Initially, she brushes a hand through her hair before raising both hands to rest behind her head, elbows pointed outwards. She then begins to sway her hips and upper body from side to side in a steady, rhythmic dance. Her movement is fluid and confident, centered on her core. The camera remains completely static in a medium shot, keeping the woman centered in the frame. The background, marked by vertical shadow lines from the sun, is completely still, highlighting the subject's gentle, looping motion.

Target Caption: A young man with short dark hair stands in front of a glowing city skyline at dusk, the sky fading into deep blue behind him. He wears a dark short-sleeve shirt with a pair of sunglasses hanging from the collar, along with a black backpack resting on both shoulders. At first, he stands relaxed, facing the camera with a calm expression. He then lifts both hands and places them loosely behind his head, elbows angled outward. As he holds this posture, he begins to gently sway his upper body from side to side in a slow, rhythmic motion, centered around his torso rather than his legs. The movement is smooth and unhurried, giving a sense of casual confidence. Throughout the sequence, the camera remains completely static in a medium shot, keeping the man clearly framed against the distant, sparkling city lights, which stay motionless and serene in the background.

Target Video:

wan_vap_demo.mp4

• Suggested Solution: We encourge to first try the semantic similar to our dataset. If you are interested in a specific domain and the domain semantic is a little far from our dataset, it would be better to have a little training to adapt (E.g., LoRA).

如果是科目三，青海摇，之类舞蹈能够支持吗？

[gangxu822]

这个模型必须配以复杂的prompt，如“图片说明：
一位留着棕色长发、戴着太阳镜的年轻女子站在一面波纹金属墙上色彩斑斓的涂鸦壁画前。她身穿白色紧身背心和浅色破洞牛仔裤。起初，她用手拨弄了一下头发，然后举起双手放在脑后，手肘向外。接着，她开始以稳定而有节奏的方式左右摇摆臀部和上半身。她的动作流畅自信，重心集中在核心部位。镜头保持静止，以中景拍摄，使女子始终位于画面中心。背景中，阳光投下的垂直阴影线清晰可见，完全静止，突显了人物轻柔而流畅的舞动。

目标画面描述：
一位留着深色短发的年轻男子站在暮色中灯火辉煌的城市天际线前，身后的天空渐渐染成深蓝色。他身穿深色短袖衬衫，领口挂着一副太阳镜，双肩背着黑色背包。起初，他姿态放松，面带平静地面向镜头。随后，他抬起双手，轻轻地放在脑后，手肘略微外展。保持这个姿势，他开始以躯干而非腿部为中心，缓慢而有节奏地左右摇摆上半身。动作流畅自然，给人一种轻松自信的感觉。整个过程中，镜头始终保持静止，以中景镜头拍摄，将男子清晰地置于画面中央，远处闪烁的城市灯光则静静地矗立在背景中，宁静祥和。”。才能生成比较好的效果吗？

[yxbian23]

这个模型必须配以复杂的prompt，如“图片说明： 一位留着棕色长发、戴着太阳镜的年轻女子站在一面波纹金属墙上色彩斑斓的涂鸦壁画前。她身穿白色紧身背心和浅色破洞牛仔裤。起初，她用手拨弄了一下头发，然后举起双手放在脑后，手肘向外。接着，她开始以稳定而有节奏的方式左右摇摆臀部和上半身。她的动作流畅自信，重心集中在核心部位。镜头保持静止，以中景拍摄，使女子始终位于画面中心。背景中，阳光投下的垂直阴影线清晰可见，完全静止，突显了人物轻柔而流畅的舞动。

目标画面描述： 一位留着深色短发的年轻男子站在暮色中灯火辉煌的城市天际线前，身后的天空渐渐染成深蓝色。他身穿深色短袖衬衫，领口挂着一副太阳镜，双肩背着黑色背包。起初，他姿态放松，面带平静地面向镜头。随后，他抬起双手，轻轻地放在脑后，手肘略微外展。保持这个姿势，他开始以躯干而非腿部为中心，缓慢而有节奏地左右摇摆上半身。动作流畅自然，给人一种轻松自信的感觉。整个过程中，镜头始终保持静止，以中景镜头拍摄，将男子清晰地置于画面中央，远处闪烁的城市灯光则静静地矗立在背景中，宁静祥和。”。才能生成比较好的效果吗？

Thanks for the thoughtful question! 🙏

1. Regarding complex / high-motion dance styles
   These types of dances can be challenging to reproduce well. The underlying backbones we rely on (CogVideoX / Wan2.1) currently do not generate such high-tempo, large-amplitude dance motions very reliably, and our training dataset does not include this category of dance (mainly because collecting and annotating high-quality examples is quite costly).
   So while we haven’t specifically tested these cases, we expect the results to be unstable.
   If your goal is to reproduce a specific dance style, the most effective solution would be to perform a small-scale finetuning:

   Adding a curated subset of those dance clips and finetuning on top of our released checkpoint will significantly improve motion fidelity for that category.

2. Why such long / structured prompts are needed
   The model relies on the reference video prompt + target video prompt to understand which semantic attributes should be transferred — motion, style, concept, camera dynamics, etc.
   If only two videos are provided without prompts, the model cannot infer what you want to transfer (motion vs. style vs. concept).
   A more elegant long-term solution would be to use instruction-based control, where the user can specify intent directly (e.g., "transfer only the motion").
   As mentioned in our paper, this is planned as future work — and we’re excited about this direction.

If you’re interested in trying finetuning or exploring instruction-based control ideas, we’d love to hear more — feel free to open a discussion or PR! 🤝