Fix crash at exit and enable sim reconstruction

[eugenevinitsky]

static void setCudaHeapSize()
{
// FIXME size limit for device side malloc:
size_t heap_size = 4ul1024ul1024ul*1024ul;

char* user_heap_size = getenv("MADRONA_MWGPU_DEVICE_HEAP_SIZE");
if (user_heap_size) {
    heap_size = strtoul(user_heap_size,nullptr,10);
}

REQ_CUDA(cudaDeviceSetLimit(cudaLimitMallocHeapSize,
                            heap_size));

}

[aaravpandya]

Closing this issue as it is not currently possible due to restrictions from madrona side. Also, we have verified that the sim does not crash at exit. Crashes at exit were mostly artifacts of mixing different python versions.

More info on sim reconstruction here - shacklettbp/madrona#36.

[eugenevinitsky]

I'm going to reopen this and mark it low priority. I don't want to forget that this is something we want to fix.