Options for saving model objects in order to restore at same point later

[alexdewar]

From the meeting minutes:

Pickling the model:

• We want to be able to capture a simulation state in order to be able to repeatedly restart it in experiments - this could be after a burn in period or to test various interventions on the same starting point.
• At the moment, we would have to try and set it up again using the outputs but it is unlikely that enough of the model state is captured in the Data object to be able restart it exactly.
• We could increase the amount of data written to the Data to ensure that a model can be started up exactly, but that that changes the scope of the Data object and would lead to a lot of variables being written, some of which aren't well suited to DataArray formats.
• Instead, is there a programmatic way we could use to write the model objects out to file and allow them to be reloaded to restart the model? The dill package looks like it might be interesting but what are our other options: we'd like a review of possible starting points, please!

I'll scope this out and write my findings here, if that's ok.

[alexdewar]

@davidorme I'm just having a play with things now. I've tried pickling every model whenever update() is called and writing the result to file. To my surprise, it worked! (Though obvs there's more to the problem than this!)

I was assuming you suggested the dill package because the models couldn't be serialised with pickle. Have I missed something? Is it sufficient to serialise the BaseModel objects (in my current dumb implementation, this includes a Data object) or are there associated data lurking elsewhere in the program that I've missed?

[davidorme]

I ran dill on a model object and that worked - I was startled! I didn't even try pickle but then the last time I did try using pickle was a looong time ago. I remember it choking on relatively simple structures but it might be much more general now.

My intuition is that if we pickle the models, Data and Config then that should be sufficient to restart the model. I think initially we want to dump them all when a model ends. There is a fringe use case for dumping selected models at each update, but that would be to do some kind of unusual data extraction - very much not a user facing thing.