Bump optuna from 3.4.0 to 4.5.0

[dependabot]

Bumps optuna from 3.4.0 to 4.5.0.

Release notes

Sourced from optuna's releases.

v4.5.0

This is the release note of v4.5.0.

Highlights

GPSampler for constrained multi-objective optimization

GPSampler is now able to handle multiple objective and constraints simultaneously using the newly introduced constrained LogEHVI acquisition function.

The figures below show the difference between GPSampler (LogEHVI, unconstrained) vs GPSampler (constrained LogEHVI, new feature). The 3-dimensional version of the C2DTLZ2 benchmark problem we used is a problem where some areas of the Pareto front of the original DTLZ2 problem are made infeasible by constraints. Therefore, even if constraints are not taken into account, it is possible to obtain the Pareto front. Experimental results show that both LogEHVI and constrained LogEHVI can approximate the Pareto front, but the latter has significantly fewer infeasible solutions, demonstrating its efficiency.

Optuna v4.4 (LogEHVI)	Optuna v4.5 (Constrained LogEHVI)

Significant speedup of TPESampler

TPESampler is significantly (about 5x as listed in the table below) faster! It enables a larger number of trials in each study. The speedup was achieved through a series of enhancements in constant factors.

The following table shows the speed comparison of TPESampler between v4.4.0 and v4.5.0. The experiments were conducted using multivariate=True on a search space with 3 continuous parameters and 3 numerical discrete parameters. Each row shows the runtime for each number of objectives and each column shows each number of trials to be evaluated. Each runtime is shown along with the standard error over 3 random seeds. The numbers in parentheses represent the speedup factor in comparison to v4.4.0. For example, (5.1x) means the runtime of v4.5.0 is 5.1 times faster than that of v4.4.0.

n_objectives/n_trials	500	1000	1500	2000
1	1.4 $\pm$ 0.03 (5.1x)	3.9 $\pm$ 0.07 (5.3x)	7.3 $\pm$ 0.09 (5.4x)	11.9 $\pm$ 0.10 (5.4x)
2	1.8 $\pm$ 0.01 (4.7x)	4.7 $\pm$ 0.02 (4.8x)	8.7 $\pm$ 0.03 (4.8x)	13.9 $\pm$ 0.04 (4.9x)
3	2.0 $\pm$ 0.01 (4.2x)	5.4 $\pm$ 0.03 (4.4x)	10.0 $\pm$ 0.03 (4.6x)	15.9 $\pm$ 0.03 (4.7x)
4	4.2 $\pm$ 0.11 (3.2x)	12.1 $\pm$ 0.14 (3.9x)	20.9 $\pm$ 0.23 (4.2x)	31.3 $\pm$ 0.05 (4.4x)
5	12.1 $\pm$ 0.59 (4.7x)	30.8 $\pm$ 0.16 (5.8x)	50.7 $\pm$ 0.46 (6.5x)	72.8 $\pm$ 1.13 (7.1x)

Significant speedup of plot_hypervolume_history

plot_hypervolume_history is essential to assess the performance of multi-objective optimization, but it was unbearably slow when a large number of trials are evaluated on a many-objective (The number of objectives > 3) problem. v4.5.0 addressed this issue by incrementally updating the hypervolume instead of calculating each hypervolume from scratch.

The following figure shows the elapsed times of hypervolume history plot in Optuna v4.4.0 and v4.5.0 using a four-objective problem. The x-axis represents the number of trials and the y-axis represents the elapsed times for each setup. The blue and red lines are the results of v4.4.0 and v4.5.0, respectively.

CmaEsSampler now supports 1D search space

Up until Optuna v4.4, CmaEsSampler could not handle one-dimensional space and fell back to random search. Optuna v4.5 now allows the CMA-ES algorithm to be used for one-dimensional space.

The optunahub library is available on conda-forge

Now, you can install the optunahub library via conda-forge as follows.

conda install conda-forge::optunahub

... (truncated)

Commits

• d7e1c1b Merge pull request #6251 from y0z/feature/bump-up-version
• ad24880 Bump up version to v4.5.0
• b706a63 Merge pull request #6240 from fusawa-yugo/fusawa-yugo/lru_cache_for_hssp
• aba5302 Merge pull request #6245 from nabenabe0928/hotfix/add-hv-for-zero-size
• 8054dbb Add a unit test
• eba34c9 Merge remote-tracking branch 'upstream/master' into hotfix/add-hv-for-zero-size
• a556b08 Merge pull request #6246 from nabenabe0928/hotfix-ci
• 268ebc1 Address c-bata's comment
• 86f0c4c revert
• 8e970db Fix
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)