Work with large models with only 16GB RAM

[jjrbfi]

Hello 👋!

I'm using a laptop, with 16GB of RAM and GNU/Linux.

When I wanted to convert the 13B and up models to ggml FP16 format, RAM got stuck... Obviously.
So, because I wanted to do it without paying anything in the cloud... I ended up using SWAP.

---

I used SWAP to extend the virtual memory beyond the installed physical memory (RAM). We can use our HDD/SSD/NVMe... for this.

Is not that fast as RAM but does the job! 🙂

I used my self a 40GB swap file. I didn't use swap partition. Here is a quick tutorial that can guide you to do so:
https://wiki.archlinux.org/title/Swap#Swap_file

I think this is possible to do in MacOS as well 🤗.

Image example working with 13B model: