Generate Specialized Code for known objective functions (recording prior dev discussion on this topic)

[cartazio]

One long standing idea that's been discussed is generating more specialized / optimized code when the expression to be evaluated is known.

An example could be generating specialized gradient code for a known objective function, etc.

Theres three main strategies for doing so:

a) template haskell, which would likely be a bit tricky to write and make easy to use.
b) a ghc compiler plugin, which if written well, would work for normal AD code as is, or at least will do so some fraction of the time.
c) alternatively, some sort of run time code generation scheme via llvm might be possible.