Any way to improve memory bandwidth for better ms per token for intel silicon?

[t1maccapp]

Been running inference for gpt_bigcode-santacoder-ggml.bin q4_1 on two systems:

1. Macbook Pro, m1 Pro + 16GB RAM
2. i5 12600k, 32GB RAM DDR5 5200 MHz (40-40-40)

Tried different amount of threads with ./bin/starcoder -m ../models/bigcode/gpt_bigcode-santacoder-ggml.bin -p "def fibonnaci(" --top_k 0 --top_p 0.95 --temp 0.2 -t N, here is the comparison:

I also ran mbw tests on both systems:

1. i5 12600k + ddr5 5200:

AVG  Method: MEMCPY  Elapsed: 0.18891  MiB: 2048.00000  Copy: 10841.354 MiB/s
AVG  Method: DUMB  Elapsed: 0.08703  MiB: 2048.00000  Copy: 23530.953 MiB/s
AVG  Method: MCBLOCK  Elapsed: 0.12635  MiB: 2048.00000  Copy: 16209.367 MiB/s

2. m1 Pro 16gb:

AVG  Method: MEMCPY  Elapsed: 0.05120  MiB: 2048.00000  Copy: 40003.203 MiB/s
AVG  Method: DUMB  Elapsed: 0.08666  MiB: 2048.00000  Copy: 23632.205 MiB/s
AVG  Method: MCBLOCK  Elapsed: 0.04780  MiB: 2048.00000  Copy: 42844.830 MiB/s

Questions: Why my mac is so much faster? Is it due memory bandwidth difference? (200 GB/s on mac vs 76.8 GB/s on intel). Any chance there is a way to boost inference speed on intel?

[t1maccapp]

Found a thread on reddit with answers to my questions.

Also, tried turning off Efficient cores and Hyper-threading – no changes.