New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement ] FSDP support #27

Open

devrituraj opened this issue Jun 11, 2023 · 0 comments

devrituraj commented Jun 11, 2023

Nice work !

I'm curious, do you have any plans to provide FSDP support for training the model on GPUs with limited capacity (24/40GB) ?

Also, how much time it takes to train the model on 8 - A100 GPUs( 80 GB probably used) ?

Thank You!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment