Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improved Lora finetuning script #1179

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
+12 −3
Open

Improved Lora finetuning script #1179

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0cae6b4
added code to check if val data is longer than train data which preve…
pattplatt Mar 22, 2024
e9cd2b1
Update litgpt/finetune/lora.py
rasbt Apr 16, 2024
6bf87f0
Merge branch 'main' into improve_lora_finetuning
rasbt Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions litgpt/finetune/lora.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import math
import os
import time
import random
from pathlib import Path
from pprint import pprint
from typing import Dict, List, Literal, Optional, Tuple, Union
Expand Down Expand Up @@ -234,10 +235,13 @@ def fit(
data: DataModule,
) -> None:
tokenizer = Tokenizer(checkpoint_dir)
longest_seq_length, longest_seq_ix = get_longest_seq_length(train_dataloader.dataset)
carmocca marked this conversation as resolved.
Show resolved Hide resolved

longest_seq_length_train, longest_seq_ix = get_longest_seq_length(train_dataloader.dataset)
longest_seq_length_val, longest_seq_ix = get_longest_seq_length(val_dataloader.dataset)
longest_seq_length = longest_seq_length_train if longest_seq_length_train > longest_seq_length_val else longest_seq_length_val
pattplatt marked this conversation as resolved.
Show resolved Hide resolved
model.max_seq_length = min(longest_seq_length, train.max_seq_length or float("inf"))
fabric.print(
f"The longest sequence length in the train data is {longest_seq_length}, the model's maximum sequence length is"
f"The longest sequence length in the train data is {longest_seq_length_train}, longest sequence lenght in the validation data is {longest_seq_length_val}. The model's maximum sequence length is"
rasbt marked this conversation as resolved.
Show resolved Hide resolved
f" {model.max_seq_length} and context length is {model.config.block_size}"
)

Expand Down Expand Up @@ -343,7 +347,12 @@ def validate(
val_loss = losses.mean()

# produce an example:
instruction = "Recommend a movie for me to watch during the weekend and explain the reason."
rand = random.randint(0, 50)
try:
instruction = val_dataloader.dataset.data[rand]["instruction"]
except Exception as e:
print(f"Import of validation data failed: {e}")
instruction = "Recommend a movie for me to watch during the weekend and explain the reason."
Comment on lines +351 to +356
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can understand the desire for not hardcoding the instruction. On the other hand, always using the same one is useful to observe progress in the continuation.

Maybe it's best to drop this bit entirely instead

Copy link
Author

@pattplatt pattplatt Mar 25, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was that it is helpful to get a feeling if the model generalizes well over the validation data. If you always have the same prompt you can't really tell, or am I missing something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this as a preference without no right or wrong. I'll defer this decision to the folks who finetune the most: @rasbt and @awaelchli

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say for the generalization aspect, we already calculate the loss over the validation set. The fixed prompt here is more of a small visual check, and I do think it helps having it the same prompt.

We could potentially do it like this:

  • by default select a random validation set instruction (like we do now) and keep it constant over the training for visual purposes
  • let users override this in the config perhaps via a "rotate" argument or so. Where validation_instruction: str = "fixed" defaults to the current behavior but that might be overkill

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear to me when this happens. Everytime the validation function is called?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To add what Sebastian said I would say that we can't tell very well from a single example by how much the model is improving. Whether it is a sample from the dataset or one we provide doesn't matter much. It's there as a sanity check to make sure the model eventually starts following instructions and adopting the prompt template. I am in favor of keeping it simple.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear to me when this happens. Everytime the validation function is called?

I'd say if we were to add the rotation, that would be also done every eval.interval steps (the default is 100) (which currently includes both calculating the loss over the entire validation set and then also using the one example prompt/instruction for a quick visual sanity check that the model generates coherent text.)

fabric.print(instruction)
prompt = data.prompt_style.apply(instruction)
encoded = tokenizer.encode(prompt, device=fabric.device)
Expand Down
Loading