Regarding the duplicate tag (you are probably referring to issue #151 ?) I can definitely see your point, but why not implement it and let the user decide via a configurable training script argument.

If implemented correctly, I also don't see why this would hinder the use of pruners. They could work based on mean/median objective performance of current and past trials.

Hello,

sorry for the late reply was on holidays...

Is that maybe exactly what you mention there?

Yes

Regarding the duplicate tag (you are probably referring to issue #151 ?)

yes and that comment:
#151 (comment)

can definitely see your point, but why not implement it and let the user decide via a configurable training script argument.

I would be happy to have a draft PR ;)

You should also know that this exist: #114

If implemented correctly, I also don't see why this would hinder the use of pruners

How do you prune a trial before the end a run if your objective is the mean/median of several runs?

I can definitely see your point, but why not implement it and let the user decide via a configurable training script argument.

I agree with this.
Faced with the same problem, I've already implemented a script that roughly does this. If you open a PR, I would be happy to contribute.

How do you prune a trial before the end a run if your objective is the mean/median of several runs?

By training multiple models simultaneously. Something like

# ...
for split in range(n):
    mean_rewards = []
    for model in models:
        model.learn(split_size, reset_num_timesteps=False)
        mean_reward, _ = evaluate_policy(model, eval_env)
        mean_rewards.append(mean_reward)
    median_score = np.median(mean_rewards)
    trial.report(median_score, split*split_size)

I wonder if you can run, say, 50 or so models simultaneously, without having memory problems or anything.

If you open a PR, I would be happy to contribute.

Please do =)

By training multiple models simultaneously. Something like

I was afraid of that answer... yes it does work but not for image-based environment and requires beefy machine anyway (for instance for DQN on Atari, a single model may require 40GB of RAM).
We also need to check if the model.learn(reset_num_timesteps=False) works well with schedules.

50 or so models simultaneously, without having memory problems or anything.

I would run only maximum 3-5 models simultaneously, unless the env is very simple and the network small.

Please do =)

Let's open a draft PR and continue the discussion there.