You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Fine Tuning Open Source LLM Models - QLora and Lora features implemented
2
2
3
-
(short, 1-3 sentenced, description of the project)
3
+
## Overview
4
+
Open-source LLMs are powerful but require fine-tuning for specific tasks like chatbots or content generation. Fine-tuning these models can be expensive due to the need for substantial VRAM. For instance, fully fine-tuning the Llama7B model requires 112GB of VRAM. However, techniques like QLoRA and PEFT can significantly reduce these requirements.
4
5
5
-
## Features
6
+
In this repo, we demonstrate how to fine-tune the Mistral 7B/OpenHermes model for a **text-to-SQL** task.
6
7
7
-
This project framework provides the following features:
8
+
We can load models from various sources: azure model catalog, local folder, or the Hugging Face website. The main notebook here demonstrates how to load models directly from Hugging Face. If the Azure model catalog contains the model you need, you can use it by either registering the model in your workspace or accessing it directly from the catalog registry.
8
9
9
-
* Feature 1
10
-
* Feature 2
11
-
* ...
12
10
13
-
## Getting Started
11
+
### Objectives and Features
12
+
1. Learn the typical LLM fine-tuning process.
13
+
2. Utilize **QLoRA** and **PEFT** to overcome GPU memory limitations.
14
+
3. Use **MLflow** to manage model training, logging artifacts, hyperparameters, metrics, and prompts.
15
+
4. Save prompt templates and inference parameters in MLflow for simplified prediction interfaces.
14
16
15
-
### Prerequisites
16
17
17
-
(ideally very short, if any)
18
+
#### Key Components - Main Notebook
19
+
*[teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B): A 7-billion parameter text-generation model optimized with techniques like Group-Query Attention and Sliding-Window Attention.
20
+
*[QLoRA](https://github.com/artidoro/qlora): Reduces trainable parameters and applies 4-bit quantization to minimize memory usage.
21
+
*[PEFT](https://huggingface.co/docs/peft/en/index): Simplifies the integration of optimization methods with pretrained models.
22
+
*[MLflow](https://mlflow.org/): Manages configurations, assets, and metrics during LLM training, integrated with Transformers and PEFT.
The Makefile is designed to set up a Python development environment using pyenv, conda, and poetry. Here's a step-by-step explanation of what each part does:
30
34
31
-
### Quickstart
32
-
(Add steps to get up and running quickly)
35
+
To run the Makefile included in this project, follow these steps:
36
+
37
+
1. Open a terminal and navigate to the project directory:
38
+
```sh
39
+
fine-tuning-open-source-text-generation
40
+
```
41
+
42
+
2. Run the Makefile using the `make` command:
43
+
```sh
44
+
make <target>
45
+
```
46
+
47
+
This will execute the default target specified in the Makefile. If you want to run a specific target, use:
48
+
Replace `<target>` with the name of the target you want to run.
49
+
50
+
This uses `condaenv` instead of `poetryenv` because Azure Machine Learning is built with `conda` and it can be easier to use when running code within Azure Machine Learning notebooks. If you are using VS Code and linking to the proper compute, you can modify the Makefile to use the `poetry` environment, or keep it as is, both will work properly.
33
51
34
-
1. git clone [repository clone url]
35
-
2. cd [repository name]
36
-
3. ...
52
+
53
+
### Setup your .env
54
+
55
+
If your model requires a token, it should be setin the `.env` file.
56
+
57
+
58
+
## Config
59
+
60
+
The config.py file provides a structured way to manage various configuration settings for different stages of a machine learning project, including modeling, deployment, and training. By using data classes and the pydantic library, it ensures type safety and easy loading/saving of configurations from/to YAML files.
37
61
38
62
39
63
## Demo
@@ -42,16 +66,40 @@ A demo app is included to show how to use the project.
42
66
43
67
To run the demo, follow these steps:
44
68
45
-
(Add steps to start up the demo)
69
+
1. Open the Jupyter Notebook file located in the `notebooks` folder.
70
+
2. Run all the cells in the notebook to execute the demo.
71
+
3. The results will be displayed within the notebook cells. Open up your AML workspace to analyze the results bellow:
72
+
73
+
74
+
- Open your web browser and go to the Azure Machine Learning Studio at https://ml.azure.com.
75
+
- Ensure you are in the correct Azure Machine Learning workspace where you ran your fine-tuning job.
76
+
- In the left-hand navigation pane, click on the "Jobs" tab. This will display a list of all the jobs that have been run in your workspace.
77
+
- Find the job corresponding to your fine-tuning task. You can use the search bar to filter jobs by name, or sort the list by date to find the most recent jobs.
78
+
- Click on the job name to open the job details page. This page provides comprehensive information about the job, including its status, logs, metrics, and outputs.
79
+
- In the "Outputs + logs" tab, you can find the artifacts generated by the job. This includes the trained model, configuration files, and any other files saved during the job. You can download these artifacts forfurther analysis or use themin subsequent steps.
80
+
81
+
If you have integrated MLflow with your job, you can also check the MLflow run associated with the job. This will provide additional details about the parameters, metrics, and artifacts logged during the training process. You can access the MLflow UI by clicking on the "MLflow run" link in the job details page.
82
+
83
+
84
+
Here is an example of the expected output:
85
+
86
+
87
+
88
+
89
+
90
+
91
+
## Contributing
92
+
93
+
We welcome contributions! Please read our [contributing guidelines](CONTRIBUTING.md) before submitting a pull request.
94
+
95
+
## License
96
+
97
+
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
46
98
47
-
1.
48
-
2.
49
-
3.
99
+
## Contact
50
100
51
-
## Resources
101
+
For any questions or feedback, please open an issue on GitHub.
52
102
53
-
(Any additional resources or related projects)
54
103
55
-
- Link to supporting information
56
-
- Link to similar sample
57
-
- ...
104
+
## Conclusion
105
+
This notebook demonstrates the process of fine-tuning an open-source LLM using QLoRA and PEFT, managing the training process with MLflow, and deploying the fine-tuned model using Azure ML.
0 commit comments