spconv SubMConv3d Plugin

[avichapman]

Code of Conduct

• I have read CODE OF CONDUCT and Support Guidelines before creating this Discussion post.

Contents

Hi all,

I am trying to get set up with the spconv TensorRT plugins that ship in the perception component. I wonder if I have done this right. I can't find any plugins for the SubMConv3d or similar classes. I do see some lower level stuff such as indice mappings, however.

Is there such a plugin and I am building it wrong? If not, can anyone suggest a way forward? I have a sparse 3d pointcloud Pytorch model that needs converting to TensorRT.

@amadeuszsz - Are you the right person to ask about this? If not, I apologise for bothering you.

Cheers

[amadeuszsz]

Hi @avichapman
Please check:

• class for training - https://github.com/tier4/AWML/blob/1db432b33b133325e6a26f3341c241f7af8d1b68/projects/SparseConvolution/sparse_conv.py#L326
• plugin for inference:
  • native - https://github.com/autowarefoundation/autoware_universe/blob/e196747bf3426fcc086ab97e7cb60863c8b44694/perception/autoware_tensorrt_plugins/src/get_indices_pairs_plugin_creator.cpp
  • non-native - https://github.com/autowarefoundation/autoware_universe/blob/e196747bf3426fcc086ab97e7cb60863c8b44694/perception/autoware_tensorrt_plugins/src/get_indices_pairs_implicit_gemm_plugin_creator.cpp

[avichapman]

Thank you for your prompt reply.

I think I may be too much of a newbie here. I don't understand how to get from the class for training to the plugin for inference.

Are there any existing projects that might help me to understand that process? I do understand that the training class under the hood calls get_indices_pairs, but I don't understand how to trace the network and then break down the graph to a level of fidelity that the calls to get_indices_pairs are exposed.

[amadeuszsz]

When you deploy to ONNX, it should figure as an operator in graph. Please check tier4/AWML#45 and autowarefoundation/autoware_universe#10600 for reference.

[avichapman]

Thank you for your information. I believe I now have my head wrapped around how to do this. I've successfully used torch2trt to create a tensorrt network using sparse operations. However, I have a problem in that get_indice_pairs only supports submanifold operations. I know that that code path has not been tested. I would like to enable non-submanifold operations. Do you know what issues I will have to solve in order to make that happen?

[amadeuszsz]

I would like to enable non-submanifold operations.

static int get_indice_pairs(ExternalAllocator& allocator, tv::Tensor indices, int batch_size, std::vector<int> input_dims, int algo, std::vector<int> ksize, std::vector<int> stride, std::vector<int> padding, std::vector<int> dilation, std::vector<int> out_padding, bool subm, bool transposed, std::uintptr_t stream_int = 0, int num_out_act_bound = -1, int num_input_act_bound = -1);

You can set subm to false.