Skip to content

Commit

Permalink
docs: more wip stuff on finetuning doc
Browse files Browse the repository at this point in the history
  • Loading branch information
ErikBjare committed Nov 5, 2023
1 parent dad5a25 commit 32df015
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions docs/finetuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Finetuning

NOTE: this document is a work in progress!

This document aims to provide a step-by-step guide to finetuning a model on conversations from gptme using the Hugging Face transformers library.
This document aims to provide a step-by-step guide to finetuning a model on conversations from gptme.

The goal of fine-tuning a model for gptme is to:

Expand All @@ -26,6 +26,8 @@ We collect our own conversation history by running the following command:

This will create files `train.csv` and `train.jsonl` in the `train` directory.

TODO: describe how to get the OpenAssistant dataset
TODO: describe how to use exported ChatGPT conversations

## Step 2: Prepare the data

Expand All @@ -37,10 +39,8 @@ We need to prepare the data for fine-tuning. This involves:

TODO...


## Step 3: Fine-tune the model


Options:

- [axolotl][axolotl]
Expand All @@ -50,7 +50,7 @@ Options:
- [OpenPipe][openpipe]?
- Looks interesting, but not sure if it's relevant for us.

TODO..
TODO...

## Model suggestions

Expand Down

0 comments on commit 32df015

Please sign in to comment.