Save cached latents as caching progresses

[SrGonao]

This is a complicated change. At the moment, we are holding all feature activations in RAM while caching, which becomes problematic when dealing with millions of tokens.
I thing the way we want to do this is to use something like huggingface datasets.

[kernel-loophole]

@SrGonao would love to work on this ,can you provide more details about it .

[SrGonao]

Currently, we do feature caching by keeping the activations in memory, before saving it (https://github.com/EleutherAI/sae-auto-interp/blob/v0.2/sae_auto_interp/features/cache.py#L208-L242). We could potentially keep saving it after X amount of tokens and then merge them at the end. This would allow for people to do longer runs where feature activations don't all fit in memory

[kernel-loophole]

okay great .will look into that .how can i test this approach