v0.5.3

What's Changed

• Post-release setup for 0.5.3.dev1 by @rasbt in #1799
• Add Phi 3 128k model by @deveworld in #1800
• Add token counts to compute performance by @rasbt in #1801
• Fixed the issue that precision is always "32-true". by @jianpingw in #1802
• Add Nvidia Llama 3.1 70B Nemotron weights by @rasbt in #1803
• Choose evaluation example from test set by @rasbt in #1804
• Pretrain tok sec by @rasbt in #1805
• typo in convert_to_litgpt command by @wasifferoze in #1807
• Move distributed all_reduce import into a function by @IvanYashchuk in #1810
• Remove hardcoded 32-precision conversion by @rasbt in #1814

New Contributors

• @deveworld made their first contribution in #1800
• @jianpingw made their first contribution in #1802
• @wasifferoze made their first contribution in #1807
• @IvanYashchuk made their first contribution in #1810

Full Changelog: v0.5.2...v0.5.3

v0.5.2

What's Changed

• Fix prompt style usage in evaluation example by @rasbt in #1790
• Pin to PyTorch 2.4.1 by @rasbt in #1796
• Fix step iteration bug in finetuning scripts by @rasbt in #1794
• Housekeeping: remove unused code and fix tests by @rasbt in #1795

Full Changelog: v0.5.1...v0.5.2

v0.5.1

What's Changed

• v0.5.0 post release setup by @rasbt in #1774
• Be more specific about missign RoPE parameters by @rasbt in #1781
• Use correct Llama 3.1 and 3.2 context lengths by @rasbt in #1779
• Fixing Llama 3.1 and 3.2 Maximum Context Length by @rasbt in #1782
• Use more realistic RoPE tests by @rasbt in #1785
• AMD (MI250X) support by @TensorTemplar in #1775
• Tidy up RoPE by @rasbt in #1786
• Bump version for 0.5.1 bugfix release by @rasbt in #1787

New Contributors

• @TensorTemplar made their first contribution in #1775

Full Changelog: v0.5.0...v0.5.1

v0.5.0

What's Changed

• Post 0.4.13 release set up by @rasbt in #1755
• Add missing explanation on how to use a finetuned model by @rasbt in #1756
• Bump lightning version to latest stable release (2.4.0) by @rasbt in #1765
• Improve rope by @rasbt in #1745
• Add bnb.nn.StableEmbedding for quantized training by @rasbt in #1770
• [fix][1760] Added fix for the missing context key issue in dolly! by @pytholic in #1766
• Fix Llama 3.2 tokenizer by @rasbt in #1772

New Contributors

• @pytholic made their first contribution in #1766

Full Changelog: v0.4.13...v0.5.0

v0.4.13

What's Changed

• Make 0.4.13.dev1 version by @rasbt in #1722
• Enable MPS support for LitGPT by @rasbt in #1724
• Simplify MPS support by @rasbt in #1726
• Add Chainlit Studio by @rasbt in #1728
• Fixing the tokenizer for slimpajama data preparation by @tomaslaz in #1734
• Add pretrain conversion by @rasbt in #1735
• Typo fix and formatting improvements in API Trainer docs by @rasbt in #1736
• bump macos to m1 by @t-vi in #1725
• Improve filepath handling in unit tests by @rasbt in #1737
• Add a more informative message in case text exceeds context size by @rasbt in #1738
• Update Thunder README.md by @rasbt in #1740
• Add sliding window attention to Mistral and Phi 3 by @rasbt in #1741
• Extend context length for sliding window tests by @rasbt in #1742
• Fix jsonarparse version by @rasbt in #1748
• Update RoPE tests by @rasbt in #1746
• Make json parsing more robust by @rasbt in #1749
• Support for optimizers which don't have "fused" parameter such as grokadamw and 8bit bnb by @mtasic85 in #1744
• Increase rtol and atol in Gemma 2 for macOS by @rasbt in #1751
• Repair json files by @rasbt in #1752
• Llama 3.2 weights by @rasbt in #1750
• Bump version to 0.4.13 for new release by @rasbt in #1753
• Temporarily take out thunder dependency for deployment by @rasbt in #1754

New Contributors

• @tomaslaz made their first contribution in #1734
• @mtasic85 made their first contribution in #1744

Full Changelog: v0.4.12...v0.4.13

v0.4.12

What's Changed

• Preserve eos in encoding when max_seq_length = -1 by @sanderland in #1694
• Add batched_next_token() and batched_sample() by @apaz-cli in #1693
• bump thunder dependency to main by @t-vi in #1701
• add support for batched input_pos to model by @t-vi in #1700
• Add batched_generate_fn() by @apaz-cli in #1702
• minor Readme update/typos by @Borda in #1710
• Fix device Error in Decode Stream by @Motsepe-Jr in #1707
• Temporarily remove MPS device support by @rasbt in #1718
• Add uvloop as a dependency by @rasbt in #1719
• Change default precision on macOS by @rasbt in #1720
• Bump for v0.4.12 by @rasbt in #1721

New Contributors

• @Motsepe-Jr made their first contribution in #1707

Full Changelog: v0.4.11...v0.4.12

v0.4.11

What's Changed

• Add distribute=None to python-api.md by @rasbt in #1676
• Make LitGPT LLM API compatible with PyTorch Lightning Trainer 1/2 by @rasbt in #1667
• Auto device handling in LLM API by @rasbt in #1677
• Fix KV cache issue in LLM API by @rasbt in #1678
• Improved benchmark utils by @rasbt in #1679
• Add PR benchmark util for internal use by @rasbt in #1680
• Added git hash to benchmark utility. by @apaz-cli in #1681
• Spelling fix by @rasbt in #1685
• Add Microsoft Phi 3.5 checkpoint by @rasbt in #1687
• Update check_nvlink_connectivity by @sanderland in #1684
• Make number of generated tokens consistent with CLI by @rasbt in #1690
• Avoid error when executing benchmark util outside a git folder by @rasbt in #1691
• Combine generate() functions by @apaz-cli in #1675
• Bumb version to 0.4.11 by @rasbt in #1695
• Fix falcon prompt template by @rasbt in #1696

New Contributors

• @sanderland made their first contribution in #1684

Full Changelog: v0.4.10...v0.4.11

v0.4.10

What's Changed

• Support Tensor Parallel in Python API by @rasbt in #1661
• Swap old Llama model with Phi-3 by @rasbt in #1666
• Update azure-gpu-test.yml by @rasbt in #1669
• Support the refactored API in litgpt serve by @rasbt in #1668
• Multi-gpu serving by @rasbt in #1670
• Add Mistral Large 123B by @rasbt in #1673
• Bumb version to 0.4.10 for next release by @rasbt in #1674

Full Changelog: v0.4.9...v0.4.10

v0.4.9

What's Changed

• Update LitServe version and tests by @rasbt in #1654
• Support for using large models in the Python API via sequential generation by @rasbt in #1637
• Add a PyTorch Lightning example by @rasbt in #1656
• Refactor Python API to introduce new distribute method (part of a larger refactor for PTL support) by @rasbt in #1657
• Fix some issues with circular and relative imports by @rasbt in #1658
• Optionally return benchmark info in Python API by @rasbt in #1660
• Bumb version for 0.4.9 release by @rasbt in #1664

Full Changelog: v0.4.8...v0.4.9

v0.4.8

What's Changed

• Adds unit test to test for parity between streaming and non-streaming API by @rasbt in #1650
• Add Gemma 2 2B by @rasbt in #1651
• Pin litserve version by @rasbt in #1652
• Version bumb for Gemma 2 2B release by @rasbt in #1653

Full Changelog: v0.4.7...v0.4.8