CPU Backend

[julien-blanchon]

Any plan for support of CPU backend, typically following the examples/relu terminology, I would like to have a relu-cpu and a backend= "cpu" in the build.toml.

So I can have relu.relu run on cpu tensors.

In case of relu for example the relu-cpu can just be sugar on top of F.relu.

[danieldk]

Great question! I've gone back and forth on it. Are there examples of some really good CPU kernels would be interesting?

[julien-blanchon]

I think relu is good example, it would make no sence to recode the kernel for CPU (aten::relu is likelly enought), but it's good quality of life to have full coverage. That's why I'm talking about sugar only.

Having the possibilities to either give an C++ Aten implementation, or just a Python pytorch one would be nice to have.

So we can be sure that ops.relu(out, x) is gonna pass on every backend

[julien-blanchon]

Nevertheless I found this repo with quite a lot of custom write kernel + cpu binding: https://github.com/aphrodite-engine/aphrodite-engine/blob/0dc79879fdc2530f801d5794d9602300851b2774/kernels/cpu/torch_bindings.cpp#L69

Seems to be OpenMP

More kernel here: https://grep.app/search?q=torch%3A%3AkCPU