poor performance on cuda

[liblaf]

I implemented a naive Adam solver and tested it on CPU and Cuda. It turns out that the calculations on my laptop CPU are even faster than on the RTX 3090! However, the kernel time on Cuda is indeed much lower than on CPU (although still not up to my expectations).

Maybe it is a problem with my implementation.

Source code and test data can be found here.

[bobcao3]

Just to do some sanity check... Can you try this code on the Vulkan backend as well?

[liblaf]

@bobcao3 I retested the code (s1 -> t1 task) on some different backends and here are the results. These results look weird to me. Could the poor performance on cuda be due to a CPU bottleneck?

Another thing I don't quite understand is why the kernel time is much shorter than actual computation time. Outside of @ti.kernel, this code is pretty much just calling the kernel in a loop. If I remove the call to the kernel, the loop takes less than 1 second to finish. However, if I keep the with ti.ad.Tape(loss): statement and re-run without any calculations (no forward and backward calculations, only pass in the with code block). It still takes a long time (0:01:16.513727 on local machine and 0:00:25.015373 on remote server).

On Local Machine (i7-1260P, no dedicated GPU)

Arch	Time	Compile Time	Kernel Time
x64	0:01:25.128130	325.730 ms	46.535 s
vulkan	0:20:01.814735	14.127 ms	Not Available

On Remote Server (E5-2683 v3, RTX 3090)

Arch	Time	Compile Time	Kernel Time
cuda	0:01:25.572995	567.996 ms	8.679 s
vulkan	0:02:02.939205	41.185 ms	Not Available
x64	0:04:35.204867	633.295 ms	189.465 s

[liblaf]

close as stale

[xiaobo-lab]

I tried too, why vulkan so fast? I have been using cuda，it is too slow!