Why "No code changes are needed" with zero-offload? What is the most basic principle here?

[chansonzhang]

e.g., How does zero-offload know the model structure and which part of params/memory to offload?

[tjruwase]

In the most basic form zero-offload offloads the entire optimizer state. These are essentially tensors regardless of the optimizer (e.g., Adam) and independent of model structure. For more details, please see the paper: https://arxiv.org/abs/2101.06840.