Skip to content

Fix for Robust Estimation Memory Issue #548

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jul 6, 2024
Merged

Fix for Robust Estimation Memory Issue #548

merged 11 commits into from
Jul 6, 2024

Conversation

azane
Copy link
Contributor

@azane azane commented Jul 5, 2024

Addresses memory issue stemming from vmap over torch.func.jvp in MonteCarloInfluenceEstimator. Instead, uses reverse mode autodiff for Jacobian of functional (largely because parameter dimensionality will typically far exceed dimensionality of functional) and then manually right multiplies param_eif (the fisher matrix X data log probability). Right multiplication is performed agnostically wrt both pytree structures and tensor shapes (emulating torch.func.jvp, with slightly more agnosticity actually).

Memory use is orders of magnitude lower, to the point of not being noticeable.

One possible difference (/cause of original problem): the vmap over jvp was potentially estimating and computing the jacobian separately for each batch in param_eif. This is very redundant, but also meant each batch saw different randomness in the Jacobian estimate, thereby propagating some notion of variability in the Jacobian estimate to the user. This implementation estimates/computes the Jacobian once only for all batches in param_eif. This may or may not be desirable, but it's important to note that doing so separately for each batch comes at very high computational cost.

Adds tests for the alternative jvp implementation, including a test of memory consumption.

@azane azane added status:WIP Work-in-progress not yet ready for review module:robust labels Jul 5, 2024
@azane azane self-assigned this Jul 5, 2024
@azane azane requested a review from eb8680 July 5, 2024 15:34
@azane azane added status:awaiting review Awaiting response from reviewer and removed status:WIP Work-in-progress not yet ready for review labels Jul 5, 2024
Copy link
Contributor

@eb8680 eb8680 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice sleuthing! Great to see such big performance gains.

@eb8680 eb8680 merged commit be862d1 into master Jul 6, 2024
7 checks passed
@eb8680 eb8680 deleted the az-robust-memory-fix branch July 6, 2024 02:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module:robust status:awaiting review Awaiting response from reviewer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants