Memory leak when regridding multiple times #424
Description
I have found out an interesting thing when using a XESMF regridder in a loop. I noticed that despite closing all the datasets with ds.close() as well as using gc.collect() there is a memory leak and the memory use accumulates with every loop run. This is a massive problem for resourcing, so I investigated a little. The solution seems to be to the following:
ds.close()
dr_out.close()
del ds ### after closing all the datasets delete the variables which will not be used in the next loop run
del regridder
del dr_out ### This is particularly important because it causes the largest memory leak.
gc.collect() ### run garbage collection at the end of each loop run, which will free the remaining memory and restore back to the levels from before the loop.
Without deleting the variables (although I always closed the datasets), the memory accumulated throughout the whole loop.