Releases: learning-at-home/lean_transformer
Releases · learning-at-home/lean_transformer
alpha 0.3.1
Initial release that matches pip_install lean_transformer==0.3.1
on PyPI
- LeanTransformer supports standard transformer architecture with optional rotary embeddings, gating, various normalization types
- Memory-efficeint backward, various parameter sharing strategies, adapters, checkpoints, reversible layers, sparsity
- LeanGPT* and LeanAlbert* models