Improved Lora finetuning script #1179
Conversation
…nts errors if this case occurs, also added feature to use val data in model training
| rand = random.randint(0, 50) | ||
| try: | ||
| instruction = val_dataloader.dataset.data[rand]["instruction"] | ||
| except Exception as e: | ||
| print(f"Import of validation data failed: {e}") | ||
| instruction = "Recommend a movie for me to watch during the weekend and explain the reason." |
There was a problem hiding this comment.
I can understand the desire for not hardcoding the instruction. On the other hand, always using the same one is useful to observe progress in the continuation.
Maybe it's best to drop this bit entirely instead
There was a problem hiding this comment.
My thought was that it is helpful to get a feeling if the model generalizes well over the validation data. If you always have the same prompt you can't really tell, or am I missing something?
There was a problem hiding this comment.
I see this as a preference without no right or wrong. I'll defer this decision to the folks who finetune the most: @rasbt and @awaelchli
There was a problem hiding this comment.
I'd say for the generalization aspect, we already calculate the loss over the validation set. The fixed prompt here is more of a small visual check, and I do think it helps having it the same prompt.
We could potentially do it like this:
- by default select a random validation set instruction (like we do now) and keep it constant over the training for visual purposes
- let users override this in the config perhaps via a "rotate" argument or so. Where
validation_instruction: str = "fixed"defaults to the current behavior but that might be overkill
There was a problem hiding this comment.
It's not clear to me when this happens. Everytime the validation function is called?
There was a problem hiding this comment.
To add what Sebastian said I would say that we can't tell very well from a single example by how much the model is improving. Whether it is a sample from the dataset or one we provide doesn't matter much. It's there as a sanity check to make sure the model eventually starts following instructions and adopting the prompt template. I am in favor of keeping it simple.
There was a problem hiding this comment.
It's not clear to me when this happens. Everytime the validation function is called?
I'd say if we were to add the rotation, that would be also done every eval.interval steps (the default is 100) (which currently includes both calculating the loss over the entire validation set and then also using the one example prompt/instruction for a quick visual sanity check that the model generates coherent text.)
Added code to check if val data is longer than train data which prevents errors if this case occurs. Also added code that allows to use of the validation data in model training except for the example prompt. These two changes could also be implemented in the other finetuning/training scripts.