ReverseDiff support

[mohdibntarek]

This PR implements ReverseDiff support which uncovered a spooky bug. Here is a reproducer that gives a wrong gradient. I will explain the spooky part and how to get the correct gradient in a comment.

using Revise, SimpleChains, ForwardDiff, ReverseDiff

x = [0.7, -0.5]
p = [-0.3, 0.6]
sc = SimpleChain(static(2), TurboDense{false}(identity, 1))
f(p) = SimpleChains.call_chain(sc, x, p)[1]
f(p)
f(p)

ForwardDiff.gradient(f, p)
ReverseDiff.gradient(f, p)

[mohdibntarek]

If you comment this line out, you get the correct gradient. deepcopying everything didn't fix this so I am not sure what's going on.

[mohdibntarek]

So call it once, it works. Call it twice, it breaks.

[chriselrod]

deepcopy isn't likely to help on pointers.

[mohdibntarek]

perhaps the _rrule function should be a struct which stores the initial pointer address and starts from there every time it's called

[chriselrod]

I would suggest taking this approach:
https://github.com/JuliaSIMD/LoopVectorization.jl/blob/8fe27b9014f16b356e09d0b108968402253fde90/src/LoopVectorization.jl#L265C1-L268

That is, in Julia versions without package extensions, LoopVectorization.jl loads the extension files.
In versions with package extensions, they'll instead be loaded lazily as an extension (while still benefiting from precompilation).

I think it'd make sense to give StaticArrays and ForwardDiff that same treatment, but that is obviously unrelated to this PR.

[mohdibntarek]

yes I will do that as soon as it works well

[chriselrod]

Note that vec allocates if !isa(y,AbstractVector)

This looks performance sensitive. Could use an @inbounds?

But also, why?
This seems internal. We should probably only be rruling relatively high level SimpleChains calls, that hide things like params getting called?

[mohdibntarek]

It was called in the log prior calculation.