Add support for `CallbackParam`

[langestefan]

In order to simplify the user interface to model parameters, cvxpy supports CallbackParam.

This is purely a convenience function, because functions of parameters can always be computed in the user space.

Currently, cvxpygen does not support CallbackParam and so models of this form will not compile:

import cvxpy as cp
import numpy as np
from cvxpygen import cpg

# setup model
m, n = 3, 2
x = cp.Variable(n, name="x")
A = cp.Parameter((m, n), name="A", sparsity=((0, 0, 1), (0, 1, 1)))
b = cp.Parameter(m, name="b")
d = cp.Parameter(m, name="d")
bd = cp.CallbackParam(
    shape=(m,), callback=lambda: d.value * b.value, name="bd"
)
problem = cp.Problem(cp.Minimize(cp.sum_squares(A @ x - bd)), [x >= 0])

# set values for parameters
np.random.seed(0)
A.value = np.zeros((m, n))
A.value[0, 0] = np.random.randn()
A.value[0, 1] = np.random.randn()
A.value[1, 1] = np.random.randn()
b.value = np.random.randn(m)
d.value = np.random.randn(m)

# solve
problem.solve()

# code gen
cpg.generate_code(problem, code_dir="nonneg_LS", solver="SCS")

Trace:

PS C:\dev\warmteplanner\warmteplanner-embedded> uv run .\callback_param_test.py
C:\dev\warmteplanner\warmteplanner-embedded\.venv\Lib\site-packages\cvxpy\expressions\leaf.py:478: RuntimeWarning: Writing to a sparse CVXPY expression via `.value` is discouraged. Use `.value_sparse` instead
  warnings.warn('Writing to a sparse CVXPY expression via `.value` is discouraged.'
C:\dev\warmteplanner\warmteplanner-embedded\.venv\Lib\site-packages\cvxpy\expressions\leaf.py:480: RuntimeWarning: Accessing a sparse CVXPY expression via a dense representation. Please report this as a bug to the CVXPY Discord or GitHub.
  self.save_value(self._validate_value(val))
C:\dev\warmteplanner\warmteplanner-embedded\.venv\Lib\site-packages\cvxpy\expressions\leaf.py:467: RuntimeWarning: Reading from a sparse CVXPY expression via `.value` is discouraged. Use `.value_sparse` instead
  warnings.warn('Reading from a sparse CVXPY expression via `.value` is discouraged.'
Generating code with CVXPYgen ...
Traceback (most recent call last):
  File "C:\dev\warmteplanner\warmteplanner-embedded\callback_param_test.py", line 29, in <module>
    cpg.generate_code(problem, code_dir="nonneg_LS", solver="SCS")
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dev\warmteplanner\warmteplanner-embedded\.venv\Lib\site-packages\cvxpygen\cpg.py", line 73, in generate_code
    write_c_code(problem, configuration, canon, canon_gradient, canon_solver, solver_interface, gradient_interface)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dev\warmteplanner\warmteplanner-embedded\.venv\Lib\site-packages\cvxpygen\cpg.py", line 583, in write_c_code
    write_file(os.path.join(configuration.code_dir, 'problem.pickle'), 'wb',
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               lambda x, y: pickle.dump(y, x),
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               cp.Problem(problem.objective, problem.constraints))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\dev\warmteplanner\warmteplanner-embedded\.venv\Lib\site-packages\cvxpygen\utils.py", line 22, in write_file
    function(file, *args)
    ~~~~~~~~^^^^^^^^^^^^^
  File "C:\dev\warmteplanner\warmteplanner-embedded\.venv\Lib\site-packages\cvxpygen\cpg.py", line 584, in <lambda>
    lambda x, y: pickle.dump(y, x),
                 ~~~~~~~~~~~^^^^^^
_pickle.PicklingError: Can't pickle <function <lambda> at 0x0000022B9A5714E0>: attribute lookup <lambda> on __main__ failed

[langestefan]

I have a better understanding of what's happening now. There is no issue with the cp.CallbackParam itself. The error is coming from the fact we can't pickle an anonymous function, so this code where callback is named now runs fine:

def bd_callback():
    return d.value * b.value

bd = cp.CallbackParam(shape=(m,), callback=bd_callback, name="bd")

The most obvious solution is:

1. Add a check to cp.CallbackParam to avoid passing in anonymous functions, or give it an autogenerated name instead.

But that's only treating the symptoms. The real issue of course is that cvxpygen has no translation from cp.CallbackParam to a C equivalent.

For example, in the problem above, the parameters b and d dissapear from the model because they are not directly attached to it, only bd is. So in C, that results in:

int main(int argc, char *argv[]){

  // Update first entry of every user-defined parameter
  cpg_update_A(0, 0.00000000000000000000);
  cpg_update_bd(0, 2.12904667347316856052);

 ....

  return 0;
}

A better, but more involved solution would be:

2. Translating callbacks to C code, attaching the input parameters of the callback to the model, then update the callback parameter whenever one of the input parameters is updated. Essentially mimicking what happens in python.

Solution 1 is needed anyway, but also doing solution 2 could be quite convenient in my opinion. I am currently working on a POC to demonstrate this.

[SteveDiamond]

Translating callback functions into C kind of defeats the concept of CVXPYGEN. The whole idea is to simplify the code generation by ensuring that the mapping from parameters to problem data is affine.

[langestefan]

I'm not sure I agree. Having functions of parameters inside a model has been unavoidable in my experience. For example I have some kind of efficiency parameter, and a cost parameter, I would need to multiply the two to get the full cost. Efficiency is not constant because it depends on temperature, nor is cost because it depends on the market conditions. This happens in power management problems (my field).

Currently I solve this in user space, and I update the parameters myself (this works great!). The only change would be to add this part to the codegenerator, to make it slightly more convenient.

So you have:

user space params -> model params.

If we only consider model params then it will still be DPP won't it? Is there some mathematical reason why we would care about user space params?

[SteveDiamond]

I may not understand your proposal but I thought you were suggesting opening up the code generation to model arbitrary functions of parameters? Even if there is a separation between user space and model params it still adds the complexity of doing code generation for arbitrary functions. Maybe some more limited approach would be viable though.

[langestefan]

Only the part for CallbackParam.

So if you have some affine parametrized constraint like this:

$$ Ax = y $$

Where $A = [a, b]'$. Lets say $a$ is a regular parameter, and $b=A[2]$ is a callback parameter $b = c*d$. The mapping is:

$$ f(c,d) = c\cdot d \to_1 b \to_2 A[2] $$

Currently cvxpygen would treat $a$ and $b$ like regular parameters and drop $c$ and $d$ from the model. So I compute $\to_1$ myself, and let cvxpygen do $\to_2$.

I propose that we generate code for a small C function that does $\to_1$ too from the callback in CallbackParam and keep $c$ and $d$ as user facing parameters. This would be a single line and only do really simple things, like multiply and divide. From the cvxpygen perspective, nothing changes because the problem is still affine in $b$.

[maxschaller]

Two things we need to keep in mind. We want to avoid certain operations like divisions to prevent divide-by-zero faults on embedded systems. So we'd have to think carefully which type of callback parameters we would allow. Further, we'd have to decide whether callback parameters are allowed when gradient=True, since we'd have to generate code for differentiating through the callback.

[langestefan]

Yes absolutely. I think supporting multiplications would already cover a lot of usecases. You could technically also say it's the user who provides the parameter values so it's up to them to prevent doing a division by zero?

Further, we'd have to decide whether callback parameters are allowed when gradient=True, since we'd have to generate code for differentiating through the callback.

I have little experience with that aspect, but I using the chain rule once would be sufficient in this case? Would open up a bunch more different usecases for differentiating through parameters though. But if that opens up a can of worms I would be fine leaving it out for a first version / POC.

[langestefan]

Regarding serialization (pickle) error in cvxpygen, would it be okay to create a PR in cvxpy to either:

1. add names to anonymous callback functions
2. prevent anonymous functions from being passed in as callbacks

Option 1 would be unintrusive and non-breaking. It could also be done in cvxpygen but it would be uglier. If there is a better method I'm not seeing, please let me know

[langestefan]

After exploring this a bit more I've decided not to go forward with it. Parameters that enter the model through a callback are by definition not DPP. I see no trivial way to both 1) keep DPP as it is and 2) add the parameters to the model. Either the parameters are 'floating' (not part of an expression, still part of the model) or the problem is not DPP. Both are required for cvxpygen. I still want to solve the pickling issue, I will make a PR for that