Parameter optimize_beta is not working

[Valentin-Laurent]

After investigating issue #484, I realized our implementation of the optimize_beta option is incorrect, and has been from its introduction in MAPIE v0.4. See also issue #583.

Code to fix:

• In function mapie.conformity_scores.regression.BaseRegressionScore.get_bounds (the only place where beta optimization is performed), we pass a reshaped version of conformity scores to _beta_optimize, and we probably should not.

        if optimize_beta:
            beta_np = self._beta_optimize(
                alpha_np,
                conformity_scores.reshape(1, -1),
                conformity_scores.reshape(1, -1),
            )

• The implementation of _beta_optimize seems OK-ish (see below). However, we get conformity_scores.reshape(1, -1) for parameters lower_bounds and upper_bounds, and the reshape always produces arrays of shape (1, n), so the value of len(lower_bounds) is always 1.
• In _beta_optimize, there may be an issue with the type of _alpha in the enumerate.
• In _beta_optimize, we distinguish lower_bounds and upper_bounds even though this value is the same, so it could be simplified.

After fixing this, we should:

• Implement tests to make sure optimize_beta works as expected
• Verify example notebooks that are using this feature (are the results still consistent?)

The MAPIE core team is currently in discussion regarding the priority of this fix VS other ongoing developments. Contributions are welcome :)

Note: may be helpful: when using optimize_beta with a list of alphas, the prediction intervals shape is currently incorrect.

[C-BdB]

The parameter optimize_beta is removed from the examples of the documentation for version v1-alpha. PR #624
Examples need to be updated when parameter is fixed.