Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve and refine MLP tests for extensibility and A/B testing #8590

Merged
merged 1 commit into from
Jan 18, 2025

Conversation

rpsilva-aws
Copy link
Contributor

Cherry-pick of #8561

In this PR, we include various fixes, improvements and extensions, namely:

  • Exposing the MLP test to other tests (to allow us to A/B test convergences: requirement for the grad acc tests)
  • Improving the asserts, and extending to losses and outputs
  • Using the appropriate flag for the LR and log steps
  • Improving the model layout with nn.Sequential
  • Enhance the coverage, by actually utilizing the checkpointing flag, and including a sanity test for CPU
  • Decouple to simplify the A/B coverage
  • Fix imports

@rpsilva-aws rpsilva-aws marked this pull request as ready for review January 17, 2025 22:45
@rpsilva-aws
Copy link
Contributor Author

@jeffhataws @tengyifei PTAL

@tengyifei
Copy link
Collaborator

@rpsilva-aws this PR has 10 commits. Is there a reason it didn't cherrypick the single squashed commit that was merged into the master branch?

@rpsilva-aws rpsilva-aws force-pushed the rpsilva_r2.6_mlp_ab_test branch from 1ffa69d to 0dd7d63 Compare January 17, 2025 22:54
@rpsilva-aws
Copy link
Contributor Author

@tengyifei There was a clash in one of the commits, so I was individually cherry-picking each commit to validate. It wasn't an actual difference, so I just squashed it now.

@tengyifei tengyifei enabled auto-merge (squash) January 17, 2025 22:57
@tengyifei tengyifei disabled auto-merge January 17, 2025 22:57
@tengyifei
Copy link
Collaborator

Ack

@tengyifei tengyifei merged commit 19d95da into pytorch:r2.6 Jan 18, 2025
12 checks passed
@rpsilva-aws rpsilva-aws deleted the rpsilva_r2.6_mlp_ab_test branch January 18, 2025 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants