Add serialization from/to ByteStrings?

[TerrorJack]

To use compact to serialize from/to ByteStrings, currently we have to go through temporary files, which makes it a bit inconvenient. Surely in-memory serialization has a lot use cases, so it would be nice if it is supported :)

[bgamari]

Note that the interface for this would likely need to be CPS'd due to the interface of withSerializedCompact. The consumer would be expected for force the entire lazy bytestring.

[ezyang]

Is the terminal state of the compact normal form truly a ByteString, or is the ByteString simply being used as a convenient interchange format for some other API of interest?

I'm a bit nervous about adding a lazy ByteString API because, although it in principle should be possible to do, it is very tricky to do correctly. Unlike most ForeignPtrs, the lazy ByteString has to either be CPS'd (as @bgamari suggests) or setup to keep a Haskell data live (not trivial, but should be possible). But I guess we should bite the bullet and do it.

[rightfold]

In addition to bytestrings it would be useful if the library allows serializing to a region in memory given by a pointer. The primary use case for this I have in mind is shared memory with mmap.