Test database a la Hypothesis?

[mwotton]

Might it be possible to capture the state of the input to the test that failed? I know there's --falsify-replay for reproducibility, but ideally, I'd be able to capture the exact way that a test failed, generate the input off that state, and save it to a test database as something to try quickly, rather than have it spend ten minutes each time grinding through finding and shrinking a complex test case.

Obviously it's possible to just capture the actual input outside the library, but presumably there's some internal value that fully determines the smallest test case that falsify could find, and it could be fed back in as something to try? I'm willing to have a bit of a hack at implementation, but if it's all boiling down to the Word64 from prim, that would be far easier & more efficient to stash. (Obviously it would probably not be preserved as semantically interesting across versions of falsify)

https://hypothesis.readthedocs.io/en/latest/database.html

[mwotton]

(had time to think about it as my current test has been running for 80 minutes)

so one option would be to just write an alternative tasty test provider that uses falsify and caches a map of types to serialised values from generators on disk. This would solve my immediate use case, but I don't think it's the right answer in some ways - if we change the definition of the underlying generator, it's possible that it could never actually generate that value, but we'd still be providing it, and it also requires some kind of serialisation scheme.

Does falsify have that same isomorphism as Hypothesis, where there's some mapping of random bits to every generated value, even the shrunken ones? If that exists, we're basically done, can just cache it as an input to feed back into the generator and use as a first test.

[edsko]

In principle yes, although that mapping is finite only for shrunken values; but that should be enough for your use-case right? So yes, in principle it should be possible to create a database with serialized finite SampleTrees for specific shrunken counter-examples.

[mwotton]

thanks - sorry for delay, that's absolutely enough for my use case. If I get some free time I'll have a hack.