Feature request

TL;DR: from peft import get_base_model_state_dict

Hi!

I'm looking for a way to get the state dict of the base model after it has been wrapped in a PeftModel while preserving the original model's state dict keys. To the best of my knowledge, the only way this can be done right now is getting the state dict from peft_model.base_model.model and manually patching the keys by removing the .base_layer. infix and filtering our peft param keys.

A reason you wouldn't want to load the base model's state dict before wrapping it, for example, is when you are loading state dicts after FSDP wrapping your peft model.

Your contribution

I have some of this logic implemented for Torchtitan. I could repurpose some of it for a PR that handles PEFT's edge-cases a bit more gracefully (so far I've only checked my approach for LoRA).