Introduce toyllm, we can do much more things based on this gpt-2 implementation, such as speculative sampling, kv cache and so on

[shenxiangzhuang]

Hi, I' here to share my project named toyllm which is based on the gpt-2 implementation on this book. Take the gpt-2 implementation as a start point, with a 16GB GPU, I implemented some interesting algorithms from scratch, such as speculative sampling, kv cache.

Which is a wonderful journey and learn a lot from it. So I decide to share it here, help this can help you too: https://github.com/ai-glimpse/toyllm