Skip to content

Latest commit

 

History

History
121 lines (91 loc) · 4.64 KB

benchmark_onnx.md

File metadata and controls

121 lines (91 loc) · 4.64 KB

CPU Benchmark (ONNX-python)

Configuration

Data set:

Aishell1 test set , the total audio duration is 36108.919 seconds.

Tools

Install Requirements

Install ModelScope and FunASR

pip install -U modelscope funasr
# For the users in China, you could install with the command:
#pip install -U funasr -i https://mirror.sjtu.edu.cn/pypi/web/simple

Install requirements

git clone https://github.com/alibaba-damo-academy/FunASR.git && cd FunASR
cd funasr/runtime/python/utils
pip install -r requirements.txt

Recipe

test_rtf

set the model, data path and output_dir

nohup bash test_rtf.sh &> log.txt &
test_cer

set the model, data path and output_dir

nohup bash test_cer.sh &> log.txt &

Paraformer-large

Number of Parameter: 220M

Storage size: 880MB

Storage size after int8-quant: 237MB

CER: 1.95%

CER after int8-quant: 1.95%

Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni

concurrent-tasks processing time(s) RTF Speedup Rate
1 (onnx fp32) 2806 0.0777 12.9
1 (onnx int8) 1611 0.0446 22.4
8 (onnx fp32) 538 0.0149 67.1
8 (onnx int8) 210 0.0058 172.4
16 (onnx fp32) 288 0.0080 125.2
16 (onnx int8) 117 0.0032 309.9
32 (onnx fp32) 167 0.0046 216.5
32 (onnx int8) 86 0.0024 420.0
64 (onnx fp32) 158 0.0044 228.1
64 (onnx int8) 82 0.0023 442.8
96 (onnx fp32) 151 0.0042 238.0
96 (onnx int8) 80 0.0022 452.0

Intel(R) Xeon(R) Platinum 8269CY CPU @ 2.50GHz 16core-32processor with avx512_vnni

concurrent-tasks processing time(s) RTF Speedup Rate
1 (onnx fp32) 2613 0.0724 13.8
1 (onnx int8) 1321 0.0366 22.4
32 (onnx fp32) 170 0.0047 212.7
32 (onnx int8) 89 0.0025 407.0
64 (onnx fp32) 166 0.0046 217.1
64 (onnx int8) 87 0.0024 414.7

Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz 32core-64processor without avx512_vnni

concurrent-tasks processing time(s) RTF Speedup Rate
1 (onnx fp32) 2959 0.0820 12.2
1 (onnx int8) 2814 0.0778 12.8
16 (onnx fp32) 373 0.0103 96.9
16 (onnx int8) 331 0.0091 109.0
32 (onnx fp32) 211 0.0058 171.4
32 (onnx int8) 181 0.0050 200.0
64 (onnx fp32) 153 0.0042 235.9
64 (onnx int8) 103 0.0029 349.9
96 (onnx fp32) 146 0.0041 247.0
96 (onnx int8) 108 0.0030 334.1

Paraformer

Number of Parameter: 68M

Storage size: 275MB

Storage size after int8-quant: 81MB

CER: 3.73%

CER after int8-quant: 3.78%

Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz 16core-32processor with avx512_vnni

concurrent-tasks processing time(s) RTF Speedup Rate
1 (onnx fp32) 1173 0.0325 30.8
1 (onnx int8) 976 0.0270 37.0
16 (onnx fp32) 91 0.0025 395.2
16 (onnx int8) 78 0.0022 463.0
32 (onnx fp32) 60 0.0017 598.8
32 (onnx int8) 40 0.0011 892.9
64 (onnx fp32) 55 0.0015 653.6
64 (onnx int8) 31 0.0009 1162.8
96 (onnx fp32) 57 0.0016 632.9
96 (onnx int8) 33 0.0009 1098.9