[Exp Result] Unable to obtain expected results

[xdobetter]

I used the data you provided "yuliang" , but I found that the results I got from running the program were quite different from those shown in the paper. Is the main reason for this the multi-concept DreamBooth? If so, how can I adjust it to reduce the deviation?

texture_ep0100_full.mp4

[YuliangXiu]

There are several potential reasons:

• The model called by the gpt4o API may vary, so you might need to update the gpt4v_prompt.txt accordingly.
• I've noticed that even with the same gpt4v_simple.json, the GroundedSAM segmentation results can vary. Check the results after running island_all.py. If there aren't enough face pictures, the identity of final 3D face may not be well-preserved. If this occurs, try adjusting the TEXT_THRESHOLD and BOX_THRESHOLD at grounding_dino_sam.py#L308 to ensure more segmented faces and other garment parts are well maintained.

[xdobetter]

I understand. Thank you very much.
1: So, if I want to reproduce your results, I can skip Step 0 and choose to use your default preprocessed data, right?
2: I find the two parameters

PuzzleAvatar/multi_concepts/grounding_dino_sam.py

Lines 308 to 309 in e1d4617

	BOX_TRESHOLD = 0.20
	TEXT_TRESHOLD = 0.25

you gave very difficult to adjust for face. Can you give me some suggestions?

[YuliangXiu]

Yes, you can skip step 0 and use my provided proprocessed data. And there are many discussions on these parameters in issue of GroundedSAM, for example, IDEA-Research/Grounded-Segment-Anything#340

[xdobetter]

I have a question about multi-concept DreamBooth training. I found that the training time on different data is quite different, such as yuliang and yamei. Do you know if this is normal? I trained them individually using a single A6000.

[YuliangXiu]

I have a question about multi-concept DreamBooth training. I found that the training time on different data is quite different, such as yuliang and yamei. Do you know if this is normal? I trained them individually using a single A6000.

This is not normal, you should check the dataloader if both experiments share the same hyperparameters.