Is it possible to launch model on Asahi linux with large contexts like deepseek r1?

[muzzah]

Im currently getting an error loading models with large context sizes even though Im running a Mac Studio with 64 GB memory. Ive read an inherent limiitation about vulkan, is this true? I seem to be able to run gemma 12b but anything large I cant.

[rhatdan]

Are you using a podman or docker machine or trying to run it locally?

[rhatdan]

@slp Do you think there could be an issue with vulkan?

[muzzah]

Issue was resolves. Its possible to set the memory limits of Vulkan. It was reporting half the memory available (which must be standard). After updating it the model loaded.