Safe CMA-ES (GECCO 2024) #187

kento031 · 2024-08-26T02:40:08Z

This is the implementation of Safe CMA-ES [Uchida+, GECCO 2024].
https://arxiv.org/abs/2405.10534

nomuramasahir0 · 2024-11-12T02:41:57Z

@c-bata
This PR introduces a new method called SafeCMA, which will require gpytorch and torch (for Gaussian processes) when users choose to use it. I believe this PR won’t affect Optuna since SafeCMA is unrelated to Optuna, but could you please double-check to ensure there aren’t any issues?

nomuramasahir0 · 2024-11-12T03:02:57Z

README.md

 and Noisy Problems?, GECCO, 2023.](https://arxiv.org/abs/2304.03473)
 * [Nomura and Shibata 2024] [M. Nomura, M. Shibata, cmaes : A Simple yet Practical Python Library for CMA-ES, arXiv:2402.01373, 2024.](https://arxiv.org/abs/2402.01373)
 * [Ros and Hansen 2008] [R. Ros, N. Hansen, A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity, PPSN, 2008.](https://hal.inria.fr/inria-00287367/document)
+* [Uchida et al. 2024] [K. Uchida, R. Hamano, M. Nomura, S. Saito, S. Shirakawa, CMA-ES for Safe Optimization, GECCO, 2024.](https://arxiv.org/abs/2405.10534)


Could you provide a brief description and usage for SafeCMA (like LRA-CMA-ES)?

It would be clearer to explicitly state that the current implementation assumes no noise.

nomuramasahir0 · 2024-11-12T03:04:22Z

cmaes/_safe_cma.py

+
+        cov:
+            A covariance matrix (optional).
+    """


Please add the paper link like this.

nomuramasahir0 · 2024-11-12T03:11:39Z

cmaes/_safe_cma.py

+        assert c1 <= 1 - cmu, "invalid learning rate for the rank-one update"
+        assert cmu <= 1 - c1, "invalid learning rate for the rank-μ update"
+
+        cm = 1  # (eq. 54)


I suggest removing comments like eq. ~ (for Hansen's Tutorial paper) in this PR, as they could be confusing when compared with the equations in the SafeCMA paper.

On the other hand, adding equation numbers for SafeCMA-specific operations would improve clarity.

nomuramasahir0 · 2024-11-12T03:22:33Z

cmaes/_safe_cma.py

+        likelihood = gpytorch.likelihoods.GaussianLikelihood(
+            noise_constraint=gpytorch.constraints.GreaterThan(0)
+        )
+        likelihood.noise = 0


I'm curious about the validity of this setting because, as mentioned in this thread:

Some small value, but don't make it too small or numerical performance will suffer. I recommend 1e-4.
cornellius-gp/gpytorch#1196 (comment)

This implies there's room for improvement in this setting, but I'm not sure for now, so let's see how it goes.

This setting was used in some previous studies, including the followings.
https://link.springer.com/chapter/10.1007/978-3-030-58112-1_13
https://dl.acm.org/doi/10.1007/978-3-031-30229-9_51

In addition, I run the safe CMA-ES with likelihood.noise = 1e-4 on some benchmark functions. As a result, it occurred a few constraint violations that was not observed with likelihood.noise = 0. Therefore, I did not change it.

nomuramasahir0 · 2024-11-12T03:28:34Z

cmaes/_safe_cma.py

+            if out_scalar:
+                grad_norm = grad_norm.mean().to(torch.float64)
+
+            return -grad_norm


df is defined as # function that returns the norm of gradient, but it currently returns a negative value, which seems odd.
I understand this is intended for scipy.optimize.minimize, but it would be better to align it with the description.

I changed the explanation of the function as # function that returns the negative norm of gradient.

nomuramasahir0 · 2024-11-12T03:37:41Z

cmaes/_safe_cma.py

+            safe_seeds = (np.random.rand(safe_seeds_num, dim) * 2 - 1) * 5
+            safe_seeds[:, 0] = - np.abs(safe_seeds[:, 0])
+
+            # evaluation of safe seeds (with multiple safety functions)


with multiple safety functions

with a single safety function?

nomuramasahir0 · 2024-11-12T03:37:59Z

examples/safecma.py

+    safe_seeds = (np.random.rand(safe_seeds_num, dim) * 2 - 1) * 5
+    safe_seeds[:, 0] = -np.abs(safe_seeds[:, 0])
+
+    # evaluation of safe seeds (with multiple safety functions)


with multiple safety functions
with a single safety function?

nomuramasahir0 · 2024-11-12T03:41:27Z

@kento031
Thanks for the PR! I've left some comments. Could you take a look?

c-bata · 2024-11-12T04:40:08Z

cmaes/__init__.py

+try:
+    from ._safe_cma import SafeCMA  # NOQA
+except ImportError:
+    pass  # Implementation of Safe CMA-ES requires scipy, gpytorch, and torch
+


It seems that the current approach is somewhat tricky, as users may be confused if cmaes.SafeCMA doesn’t exist in certain Python environments.

How about simply removing SafeCMA from __init__.py and renaming cmaes/_safe_cma.py to cmaes/safe_cma.py?

Suggested change

try:

from ._safe_cma import SafeCMA # NOQA

except ImportError:

pass # Implementation of Safe CMA-ES requires scipy, gpytorch, and torch

Thanks for the great suggestion! It looks good to me.

@kento031
Could you make this modification if it’s okay?

kento031 · 2024-11-13T05:18:31Z

@nomuramasahir0 @c-bata
Thank you for your kind comments. I will address them.

kento031 · 2025-03-13T08:17:34Z

Sorry for being so late in addressing your comments.
I modified my implementation according to your comments.

I checked the code using act (https://github.com/nektos/act), and there remains an error when executing act -j lint (and when executing act pull_request). This error was caused at cmaes/_cma.py:480 by running mypy cmaes. I leave this error as is for now because I did not change cmaes/_cma.py.

@nomuramasahir0 We previously discussed about whether _safe_cma.py sould be contained in cmaes/__init__.py. However, at that time, I misunderstood how the workflow runs. Finally, I modified my implementation according to @c-bata 's comment.

ha-mano · 2025-03-27T11:08:32Z

@kento031
I have fixed a mypy error that was causing in the workflow tests. The fix is included in this pull request.

kento031 · 2025-03-30T05:08:03Z

@ha-mano
Thank you for fixing the error. I have merged your fix.

c-bata

Thank you for the update. LGTM with a nit.

requirements-dev.txt

Co-authored-by: c-bata <[email protected]>

c-bata

LGTM after CI passes. Thank you for your contribution 💯

kento031 · 2025-04-04T02:48:57Z

Thank you for all your comments!

In the CI check, the lint job failed due to tools/cmaes_visualizer.py and tools/ws_cmaes_visualizer.py. However, I did not change these files.

Should I modify these codes to resolve the errors?

c-bata · 2025-04-04T04:35:50Z

@kento031 No, you don't need to fix that errors in this PR 👍

safe cmaes

613567a

nomuramasahir0 requested review from c-bata and nomuramasahir0 November 12, 2024 02:36

nomuramasahir0 reviewed Nov 12, 2024

View reviewed changes

c-bata reviewed Nov 12, 2024

View reviewed changes

kento031 force-pushed the main branch 2 times, most recently from bea9914 to f5e178a Compare February 4, 2025 04:49

Update SafeCMA

d052593

kento031 force-pushed the main branch from f5e178a to 868df34 Compare February 4, 2025 04:52

Update SafeCMA

af4cc3d

kento031 force-pushed the main branch from 868df34 to af4cc3d Compare February 4, 2025 05:03

kento031 and others added 3 commits February 4, 2025 14:09

Merge branch 'main' into main

ff00b70

Update SafeCMA

f9daf74

Update SafeCMA

977a8e1

Merge branch 'CyberAgentAILab:main' into main

66a76e9

modified paper link

1f75894

c-bata reviewed Mar 31, 2025

View reviewed changes

requirements-dev.txt Show resolved Hide resolved

Update requirements-dev.txt

39c26ed

Co-authored-by: c-bata <[email protected]>

c-bata approved these changes Apr 3, 2025

View reviewed changes

c-bata merged commit f074908 into CyberAgentAILab:main Apr 4, 2025
13 of 14 checks passed

Safe CMA-ES (GECCO 2024) #187

Safe CMA-ES (GECCO 2024) #187

Uh oh!

Conversation

kento031 commented Aug 26, 2024

Uh oh!

nomuramasahir0 commented Nov 12, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nomuramasahir0 commented Nov 12, 2024

Uh oh!

c-bata Nov 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kento031 commented Nov 13, 2024

Uh oh!

kento031 commented Mar 13, 2025

Uh oh!

ha-mano commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kento031 commented Mar 30, 2025

Uh oh!

c-bata left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

c-bata left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kento031 commented Apr 4, 2025

Uh oh!

c-bata commented Apr 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

c-bata Nov 12, 2024 •

edited

Loading

ha-mano commented Mar 27, 2025 •

edited

Loading

c-bata left a comment •

edited

Loading