Open
Description
Prerequisites
Before submitting your issue, please ensure the following:
- I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
- I have carefully read and followed the instructions in the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
Feature Description
Inference of deepseek V3 at home.
Motivation
To have sonnet 3.5 level llm at home with good speed
Possible Implementation
improvement based on ktransformer with increase sparsity to make CPU inference part even faster