Trying Intel Extension for PyTorch with Silero VAD

[snakers4]

Links:

• https://github.com/intel/intel-extension-for-pytorch
• https://intel.github.io/intel-extension-for-pytorch

Basically, we tried it for an Intel CPU with AVX2 instructions.

Some details:

• Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz
• There was not pre-built binary (there was only one for AVX512), so we had to build it
• A warning UserWarning: Conv BatchNorm folding failed during the optimize process occurred, did not invest time into fixing it
• No visible tangible speed improvement:
  • For nn.Module (non-quantized)
  • For a jit container model (non-quantized)
  • Did not try FP16 or quantized INT8 models

Some caveats:

• The model is quite small
• The above issues, warnings, etc
• Our model contains both convolutions and MHSA modules

Maybe we did something wrong here.
We will trying to make use of this extension in future, but probably it will be integrated into major builds, so idk.