Training Cost

[HalvesChen]

Hi !
For your interesting work. I wonder how much resources are needed for a complete training tokenizer?

[xwen99]

Hi @HalvesChen,

Thank you for your interest in our work and sorry for my late reply. On 32 Nvidia H20 GPUs, it roughly takes 10 minutes to train the tokenizer for one epoch on ImageNet. It should roughly take 20 minutes/epoch for 8 A100s.

Best,
Xin