Problems with DDP + Optuna #639
Description
When I try to search with optuna, on 8-gpus with ddp strategy training.
The sweeper will start 8 groups of different hyperparameters, so the params shape doesn't match on each gpu.
{{ message }}
When I try to search with optuna, on 8-gpus with ddp strategy training.
The sweeper will start 8 groups of different hyperparameters, so the params shape doesn't match on each gpu.