模型参数加载错误

[GanPeixin]

使用[Google Drive]中下载的模型放到checkpoints中，运行python demo_page.py --config ./config/Dolphin.yaml --input_path ./demo/page_imgs/sample.pdf --save_dir ./results会出现下面的参数加载错误。但使用hf框架运行就没问题。

RuntimeError: Error(s) in loading state_dict for DonutModel:
        Missing key(s) in state_dict: "vpm.model.layers.3.downsample.norm.weight", "vpm.model.layers.3.downsample.norm.bias", "vpm.model.layers.3.downsample.reduction.weight".
        Unexpected key(s) in state_dict: "vpm.model.layers.0.downsample.reduction.weight", "vpm.model.layers.0.downsample.norm.weight", "vpm.model.layers.0.downsample.norm.bias", "vpm.model.layers.0.blocks.0.attn.relative_position_index", "vpm.model.layers.0.blocks.1.attn_mask", "vpm.model.layers.0.blocks.1.attn.relative_position_index", "vpm.model.layers.1.blocks.0.attn.relative_position_index", "vpm.model.layers.1.blocks.1.attn_mask", "vpm.model.layers.1.blocks.1.attn.relative_position_index", "vpm.model.layers.2.blocks.0.attn.relative_position_index", "vpm.model.layers.2.blocks.1.attn_mask", "vpm.model.layers.2.blocks.1.attn.relative_position_index", "vpm.model.layers.2.blocks.2.attn.relative_position_index", "vpm.model.layers.2.blocks.3.attn_mask", "vpm.model.layers.2.blocks.3.attn.relative_position_index", "vpm.model.layers.2.blocks.4.attn.relative_position_index", "vpm.model.layers.2.blocks.5.attn_mask", "vpm.model.layers.2.blocks.5.attn.relative_position_index", "vpm.model.layers.2.blocks.6.attn.relative_position_index", "vpm.model.layers.2.blocks.7.attn_mask", "vpm.model.layers.2.blocks.7.attn.relative_position_index", "vpm.model.layers.2.blocks.8.attn.relative_position_index", "vpm.model.layers.2.blocks.9.attn_mask", "vpm.model.layers.2.blocks.9.attn.relative_position_index", "vpm.model.layers.2.blocks.10.attn.relative_position_index", "vpm.model.layers.2.blocks.11.attn_mask", "vpm.model.layers.2.blocks.11.attn.relative_position_index", "vpm.model.layers.2.blocks.12.attn.relative_position_index", "vpm.model.layers.2.blocks.13.attn_mask", "vpm.model.layers.2.blocks.13.attn.relative_position_index", "vpm.model.layers.3.blocks.0.attn.relative_position_index", "vpm.model.layers.3.blocks.1.attn_mask", "vpm.model.layers.3.blocks.1.attn.relative_position_index".
        size mismatch for vpm.model.layers.1.downsample.norm.weight: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([512]).
        size mismatch for vpm.model.layers.1.downsample.norm.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([512]).
        size mismatch for vpm.model.layers.1.downsample.reduction.weight: copying a param with shape torch.Size([512, 1024]) from checkpoint, the shape in current model is torch.Size([256, 512]).
        size mismatch for vpm.model.layers.2.downsample.norm.weight: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1024]).
        size mismatch for vpm.model.layers.2.downsample.norm.bias: copying a param with shape torch.Size([2048]) from checkpoint, the shape in current model is torch.Size([1024]).
        size mismatch for vpm.model.layers.2.downsample.reduction.weight: copying a param with shape torch.Size([1024, 2048]) from checkpoint, the shape in current model is torch.Size([512, 1024]).

[fh2019ustc]

@GanPeixin 你好，感谢反馈~ 关于这个问题可以参考这个issue：#64