Tuning dispatch framework for discussion

[gbohner]

At the moment tuning seems to be implemented as a function that changes the parameters of a Learner. https://github.com/dominusmi/Julia-Machine-Learning-Review/blob/master/MLR/Tuning.jl

"""
    Tunes learner given a task and parameter sets.
    Returns a learner which contains best tuned model
"""
function tune(learner::Learner, task::Task, parameters_set::ParametersSet;
                sampler=Resampling()::Resampling, measure=MLMetrics.accuracy::Function,
                storage=MLRStorage()::MLRStorage)
    ...
    lrn = ModelLearner(lrn, modelᵧ, parameters_set)

    lrn

There are issues with this, the main one being, that if you apply tuning to a model, the composition of tuning-model is forgotten and only the best performing model is returned, which is of course conditional on the data available at a time. Ideally you would want to instead get a TunedLearner object, which behaves just as your original learner for all purposes, except when you fit it to data, it automatically tuned itself.

I attached a possible implementation (discussed with and approved by one of the more experienced Julia devs, Valentin Churavy), which we can use as a basis for discussing a wider framework of composable models.

[gbohner]

For some reason the github viewer doesn't show the notebook properly, you can take a quick look here:
https://nbviewer.jupyter.org/github/gbohner/Julia-Machine-Learning-Review/blob/f3482badf1f275e2a98bf7e338d406d399609f37/MLR/TuningDispatch_framework.ipynb

[dominusmi]

I was already planning on doing something like that so that's great!

Regarding the question Side design question: Should the returned model after ModelSelectionTuning be as simplified as possible, or retain its full complexity?
I've been struggling with that already, at the moment I went for simplified but that's been to simplify my life too so I'm not sure which approach is the best overall