Sampling

[dharik-arsath]

Hi, Imagine I've very big dataset and is there a way to take Stratified Sampling in vaex ml so that I'll be having less data to work with.

Note: Stratified Sampling has to be memory efficient.

[JovanVeljanoski]

There is no such option at the moment out-of-the-box. It is something to think about indeed, even for things like train/test split.

I guess if you really need it, you can do it manually: make a filtered df for a particular label, subsample, then same for the next.. and so on. At the end you can concat all the subsampled dataframes. You can export the final dataframe to gain on performance.

I will think about how to attack this properly.

Ideas/PRs are welcome!