Optimization Effort

[ferasboulala]

Seeing some traction on this project. This was implemented years ago when I did not know much about writing efficient programs. I will revisit it in the upcoming weeks for optimization and testing purposes. Interfaces may or may not change. I may also implement a GPU kernel for it.

[ferasboulala]

Looks like it will be at least 3.5x faster. 4-4.5x if the output data type that will be requested is 32 or 16 bit. Most of the time is spent doing load and stores in the forward pass.