-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Finetuning #255
Finetuning #255
Conversation
|
||
|
||
@Step.register("subset-data") | ||
class SubsetData(Step): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have the DatasetRemix
step for Tango's DatasetDict
. Can we have the same for HF's datasets?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will work on it separately: #268 This is technically unrelated to finetuning.
|
||
def run( # type: ignore[override] | ||
self, | ||
model: Lazy[Model], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I want to do some sort of curriculum learning, can I pass in the output of another training step here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be done once we fix this: #269
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And this: #270
Changes proposed in this pull request:
transformers::finetune
step, which mostly mimics theTorchTrainStep
with additions for tokenizing the data (and updating the model embeddings). It also contains model-specific defaults for the data collator.RunGeneration
now allows the trained model object as input.Before submitting
section of the
CONTRIBUTING
docs.Writing docstrings section of the
CONTRIBUTING
docs.After submitting