Why Distributed Sampler is ensuring data in a batch are of the same type?

[JeremyCJM]

Why Distributed Sampler is ensuring data in a batch are of the same type?

Lumina-mGPT/xllmx/data/sampler.py

Line 64 in 104abe4

Distributed Sampler ensuring data in a batch are of the same type (e.g. text, image-text)

[ChrisLiu6]

This is a function instead of a restriction.

If your training involves data of different kinds and, for some reason, mixing different kinds of data into one batch causes some problem, then you can assign different type attributes to different kinds of data in the data confg yaml.

In other words, if the aforementioned effect is not desired, you can simply assign different kinds of data with the same type attribute.