Skip to content

Commit f417256

Browse files
committed
update
1 parent ae14a13 commit f417256

File tree

1 file changed

+6
-0
lines changed
  • llama-index-integrations/llms/llama-index-llms-ipex-llm/examples

1 file changed

+6
-0
lines changed

llama-index-integrations/llms/llama-index-llms-ipex-llm/examples/README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,12 @@ The example [low_bit.py](./low_bit.py) shows how to save and load low_bit model
4444
python low_bit.py -m <path_to_model> -d <cpu_or_xpu> -q <query_to_LLM> -s <save_low_bit_dir>
4545
```
4646
47+
> Please note that in this example we'll use [HuggingFaceH4/zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha) model for demonstration. It requires updating `transformers` and `tokenizers` packages.
48+
>
49+
> ```bash
50+
> pip install -U transformers==4.37.0 tokenizers==0.15.2
51+
> ```
52+
4753
### More Data Types Example
4854
4955
By default, `IpexLLM` loads the model in int4 format. To load a model in different data formats like `sym_int5`, `sym_int8`, etc., you can use the `load_in_low_bit` option in `IpexLLM`. To load a model on different device like `cpu` or `xpu`, you can use the `device_map` option in `IpexLLM`.

0 commit comments

Comments
 (0)