Skip to content

mindnlp.transformers中的ConditionalDetrForObjectDetection.from_pretrained和ConditionalDetrForSegmentation.from_pretrained模型在初始化过程中未能正确创建backbone变量 #1979

Open
@4everImmortality

Description

@4everImmortality

Describe the bug/ 问题描述 (Mandatory / 必填)
ConditionalDetrConvEncoder在调用replace_batch_norm(backbone)时,backbone尚未被正确赋值。

mindspore 2.5.0 mindnlp 0.4.0
cpu
Python 3.9
Ubuntu24.04

问题复现 object detection

import matplotlib.pyplot as plt
import mindspore
from mindspore import Tensor, ops
from mindnlp.transformers import ConditionalDetrImageProcessor, ConditionalDetrForObjectDetection
from PIL import Image
import requests

# set the device to GPU if available
mindspore.set_device(device_target="GPU" if mindspore.context.get_context(
    "device_target") == "GPU" else "CPU")

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

# need to use a compatible image processor
image_processor = ConditionalDetrImageProcessor.from_pretrained(
    "microsoft/conditional-detr-resnet-50")

# need to use a compatible model for object detection
model = ConditionalDetrForObjectDetection.from_pretrained(
    "microsoft/conditional-detr-resnet-50")

报错:

Image
问题复现 segmentation

import io
import requests
from PIL import Image
from mindspore import Tensor
import numpy

from mindnlp.transformers import (
    AutoImageProcessor,
    ConditionalDetrConfig,
    ConditionalDetrForSegmentation,
)
from mindnlp.transformers.image_transforms import rgb_to_id

url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

image_processor = AutoImageProcessor.from_pretrained(
    "microsoft/conditional-detr-resnet-50")

# randomly initialize all weights of the model
config = ConditionalDetrConfig()
model = ConditionalDetrForSegmentation(config)

运行结果:

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions