`lbfgs` solver `ConvergenceWarning` during `RecursiveFeatureElimination`

[valosekj]

I got the following ConvergenceWarning warning coming from LogisticRegression's solver lbfgs:

/Users/valosek/miniconda3/envs/modular_optuna_ml/lib/python3.12/site-packages/sklearn/linear_model/_logistic.py:465: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. OF ITERATIONS REACHED LIMIT.

This was unintuitive because the default solver is set to saga:

modular_optuna_ml/testing/model_configs/log_reg.json

Line 10 in f409241

"solver": "saga",

After some in-person debugging with @SomeoneInParticular, it turned out that the warning comes from the RecursiveFeatureElimination hook, which calls LogisticRegression without passing any args, thus using the default lbfgs solver:

modular_optuna_ml/data/hooks/feature_selection.py

Line 206 in f409241

new_lor = LogisticRegression()

When I changed this line to new_lor = LogisticRegression(solver='saga'), the warning is gone. This opens the question of whether we want to allow the user to specify args for the RecursiveFeatureElimination hook.