Description
First, thanks for this awesome lib. This is a non-judgemental question, but I've been looking at issue #35 and as part of any subsequent PRs , prior to starting work would like to understand the nature of this lib.
The way I see it, there are alternatives:
- ONNX Runtime Web
- Torchscript Node Bindings
- Transformers.js
- Export to TFLite and run using TFLite web
This repo to me seems like it exists in a very thin region of deployment of a custom transformer model in pure js, without additional wasm dependencies. This is very useful for compute limited third party environments, or for integrating into a rollup bundle into a prebuilt website for example.
However the API right now is very limited no convolutional, pooling, padding, shuffling or recurrent layers. While it is possible to build a few of these layers (maybe GRU or LSTM) using existing layers, there seem other redundant layers (nn.Block = Transformer Block) which do not exist in PyTorch but exist here.
Are there any plans to:
- Support Convolutional Layers with Pooling,Padding,Shuffling etc
- Build Recurrent Layers as examples using basic building blocks already present
Either direction for this library is fine, but would like to understand what the end goal is here? Is it to provide a transformer implementation for the web (very useful for reasons stated above) or is it to provide a PyTorch API in pure Javascript (very useful again, but would require orders of magnitude more work).
Might be useful to define the scope narrowly somewhere, I've not implemented this lib in production anywhere but would like to see any production examples.