Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autocast support for half precision #174

Open
majing921201 opened this issue Jan 6, 2025 · 3 comments
Open

Autocast support for half precision #174

majing921201 opened this issue Jan 6, 2025 · 3 comments

Comments

@majing921201
Copy link

majing921201 commented Jan 6, 2025

hi, I didn't found autocast support in the train step. Where we just convert model to bfloat16.

model = model.to(torch.bfloat16)

But I found some autocast related code in hstu main body, and autocast option is hard code to "None". User cannot pass autocast argument to model.
So for my understanding autocast is not supported now, may I know do you have a plan for autocast ? Thank you

@jiaqizhai
Copy link
Contributor

Hi, the public experiments were run with autocast disabled given the datasets were small. In large-scale settings, enabling autocast would definitely be beneficial. It should be easy to flip that flag back on, but you probably want to integrate the custom triton kernels as well if efficiency is a main focus.

@majing921201
Copy link
Author

Thank you for your quick response.
So for the configs provided in current repo: https://github.com/facebookresearch/generative-recommenders/tree/main/configs. They are all not large-scale scope, right ? Is there any specific definition for "large-scale" per hstu model ?

@jiaqizhai
Copy link
Contributor

Hi - we discussed some possible industrial-scale configurations in Section 4.1.2 and Section 4.3.1 of the paper. For comparison, the Amazon Books dataset (one of the largest public datasets commonly used in prior work) consists of 694,897 users, 674,044 items, which is a few orders of magnitude smaller than the industrial-scale configurations we discussed/tested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants