Skip to content

[Bug] Possible batching bug in c++ inference api #2912

@cravies

Description

@cravies

Checklist

  • I have searched related issues but cannot get the expected help.
  • 2. I have read the FAQ documentation but cannot get the expected help.
  • 3. The bug has not been fixed in the latest version.

Describe the bug

When running a segmentation model through the c++ api I get a bug. When I run batched inference, the result returned is always the first image, multiplied by the length of the batch.
So for a size 10 batch, its just the segmentation from the first batch, stretched to length 10.
If I edit the following lines in mmdeploy\common.hpp to include the offset (editing class Result_)

-  T* operator->() const noexcept { return data_.get(); }
-  T& operator*() const noexcept { return *data_; }
+ T* operator->() const noexcept { return data_.get() + offset_; }
+ T& operator*() const noexcept { return *(data_.get() + offset_); }

It seems to fix it. Before, when we used -> to access the pointer, we were always getting the first element pointer, and when we used * to access the value we were always getting the first value. Including the offset fixes?
Therefore I think this points to a possible bug in the c++ api?

Reproduction

std::string model_path = "C:\\Users\\Ben_C\\Desktop\\mmdeploy-1.3.1\\segnext-trt\\segnext-t-fp16-batch-10";
mmdeploy::Context context;
context.Add(mmdeploy::Device("cuda"));
mmdeploy::Segmentor segmentor{ mmdeploy::Model{model_path}, context };

std::string video_path = "E:/NE_northeast.mp4";
cv::VideoCapture cap(video_path);
cv::Mat frame;
bool ret = cap.read(frame);
int count = 0;
std::vector<cv::Mat> mats;
while (ret) {
    bool ret = cap.read(frame);
    if (!ret) {
        break;
    }
    mats.push_back(frame.clone());
}

int bsize = 10;
std::vector<mmdeploy::Mat> batch;
batch.reserve(bsize);
auto start = std::chrono::high_resolution_clock::now();
std::vector<std::vector<mmdeploy::Segmentor::Result>> res;
for (int i = 0; i < mats.size(); i++) {
    batch.emplace_back(mats[i]);
    if (batch.size() == bsize) {
        check_batch(batch);
        auto seg = segmentor.Apply(batch);
        res.push_back(seg);
        batch.clear();
    }
    else if (i == mats.size() - 1) {
        auto seg = segmentor.Apply(batch);
        res.push_back(seg);
        batch.clear();
    }
    count += 1;
}

Environment

C++ api.

Error traceback

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions