Radiative transfer memory challenge

[natashabatalha]

All our calculations are memory limited because of the opacities. Specifically, for GPUs and CPUs the number of simultaneous calculations we can run is limited by 1) the number of molecules’ opacities sources you are considering, 2) the resolution of the opacities you are loading, and 3) the wavelength range. The problem is slightly different for CPUs and GPUs.

For CPUs, the opacities eventually get queried via sqlite database when requested, then summed together eventually. So the memory profile looks like a stair case as you query each molecule via get_opacities function. Then once everything is summed in compute_opacity via TAUGAS, it all gets compressed. See figure showing the memory behavior:

For GPUs it's slightly different as it is not a "get what you need" model. Instead, all the opacities must be loaded on the GPU initially so right now we are limited by the entire opacity database. So you can imagine the memory load staying high and constant throughout the calculation. Yinan Zhao has begun working on an opacity emulator that reduces the memory load of the opacity database. Will link to ongoing branch ASAP.