Skip to content

[Feature] RTMO to CoreML #3239

@adizhol

Description

@adizhol

What is the feature?

I am trying to convert RTMO to coreml using mmdeploy.
Conversions succeeds, but visualisation shows bad (empty?) boxes with coreml. Pytorch visualisation is good.

Image

pose-detection_rtmo_coreml_static.py:

ir_config = dict(type='torchscript', 
                input_shape=[640, 640],
                save_file='end2end.pt',
                input_names=['input'],
                output_names=['dets', 'keypoints'],
                )

codebase_config = dict(type='mmpose', task='PoseDetection',
                post_processing=dict(
                    score_threshold=0.05,
                    iou_threshold=0.5,
                    max_output_boxes_per_class=200,
                    pre_top_k=2000,
                    keep_top_k=50,
                    background_label_id=-1)
            )

backend_config = dict(
    type='coreml',
    convert_to='mlprogram',
    model_inputs=[
        dict(
            input_shapes=dict(
                input=dict(
                    min_shape=[1, 3, 640, 640],
                    max_shape=[1, 3, 640, 640],
                    default_shape=[1, 3, 640, 640])))
    ])

09/08 12:24:01 - mmengine - INFO - 

09/08 12:24:01 - mmengine - INFO - **********Environmental information**********
09/08 12:24:02 - mmengine - INFO - sys.platform: darwin
09/08 12:24:02 - mmengine - INFO - Python: 3.10.11 (main, May 17 2023, 14:30:36) [Clang 14.0.6 ]
09/08 12:24:02 - mmengine - INFO - CUDA available: False
09/08 12:24:02 - mmengine - INFO - MUSA available: False
09/08 12:24:02 - mmengine - INFO - numpy_random_seed: 2147483648
09/08 12:24:02 - mmengine - INFO - GCC: Apple clang version 16.0.0 (clang-1600.0.26.6)
09/08 12:24:02 - mmengine - INFO - PyTorch: 2.7.0
09/08 12:24:02 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 4.2
  - C++ Version: 201703
  - clang 15.0.0
  - OpenMP 201811
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: NO AVX
  - Build settings: BLAS_INFO=accelerate, BUILD_TYPE=Release, COMMIT_SHA=134179474539648ba7dee1317959529fbd0e7f89, CXX_COMPILER=/usr/bin/c++, CXX_FLAGS= -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_PYTORCH_QNNPACK -DAT_BUILD_ARM_VEC256_WITH_SLEEF -DUSE_XNNPACK -DUSE_PYTORCH_METAL_EXPORT -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DUSE_COREML_DELEGATE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=braced-scalar-init -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wvla-extension -Wsuggest-override -Wnewline-eof -Winconsistent-missing-override -Winconsistent-missing-destructor-override -Wno-pass-failed -Wno-error=old-style-cast -Wconstant-conversion -Qunused-arguments -fcolor-diagnostics -faligned-new -fno-math-errno -fno-trapping-math -Werror=format -DUSE_MPS -Wno-missing-braces, LAPACK_INFO=accelerate, TORCH_VERSION=2.7.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EIGEN_FOR_BLAS=ON, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=OFF, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 

09/08 12:24:02 - mmengine - INFO - TorchVision: 0.22.0
09/08 12:24:02 - mmengine - INFO - OpenCV: 4.12.0
09/08 12:24:02 - mmengine - INFO - MMEngine: 0.10.7
09/08 12:24:02 - mmengine - INFO - MMCV: 2.1.0
09/08 12:24:02 - mmengine - INFO - MMCV Compiler: clang 16.0.0
09/08 12:24:02 - mmengine - INFO - MMCV CUDA Compiler: not available
09/08 12:24:02 - mmengine - INFO - MMDeploy: 1.3.1+bc75c9d
09/08 12:24:02 - mmengine - INFO - 

09/08 12:24:02 - mmengine - INFO - **********Backend information**********
09/08 12:24:02 - mmengine - INFO - tensorrt:	None
09/08 12:24:02 - mmengine - INFO - ONNXRuntime:	None
09/08 12:24:02 - mmengine - INFO - pplnn:	None
09/08 12:24:02 - mmengine - INFO - ncnn:	None
09/08 12:24:02 - mmengine - INFO - snpe:	None
09/08 12:24:02 - mmengine - INFO - openvino:	None
09/08 12:24:02 - mmengine - INFO - torchscript:	2.7.0
09/08 12:24:02 - mmengine - INFO - torchscript custom ops:	Available
09/08 12:24:02 - mmengine - INFO - rknn-toolkit:	None
09/08 12:24:02 - mmengine - INFO - rknn-toolkit2:	None
09/08 12:24:02 - mmengine - INFO - ascend:	None
09/08 12:24:03 - mmengine - INFO - coreml:	9.0b1
09/08 12:24:03 - mmengine - INFO - tvm:	None
09/08 12:24:03 - mmengine - INFO - vacc:	None
09/08 12:24:03 - mmengine - INFO - 

09/08 12:24:03 - mmengine - INFO - **********Codebase information**********
09/08 12:24:03 - mmengine - INFO - mmdet:	3.2.0
09/08 12:24:03 - mmengine - INFO - mmseg:	None
09/08 12:24:03 - mmengine - INFO - mmpretrain:	None
09/08 12:24:03 - mmengine - INFO - mmocr:	None
09/08 12:24:03 - mmengine - INFO - mmagic:	None
09/08 12:24:03 - mmengine - INFO - mmdet3d:	None
09/08 12:24:03 - mmengine - INFO - mmpose:	1.3.2
09/08 12:24:03 - mmengine - INFO - mmrotate:	None
09/08 12:24:03 - mmengine - INFO - mmaction:	None
09/08 12:24:03 - mmengine - INFO - mmrazor:	None
09/08 12:24:03 - mmengine - INFO - mmyolo:	None

Any other context?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions