Add SUDE dimension reduction method and update related documentation #729
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull Request Overview
This pull request adds SUDE (Scalable Unsupervised Deep Embedding) as a new dimension reduction method to the Dynamo package. The implementation includes a complete SUDE algorithm with supporting mathematical functions and integrates it into the existing dimension reduction pipeline.
- Added complete SUDE algorithm implementation with supporting functions for landmark selection, embedding computation, and large dataset handling
- Integrated SUDE into the main dimension reduction API and exposed it at the package level
- Added comprehensive test coverage for SUDE functionality including edge cases and parameter validation
Reviewed Changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_sude.py | Comprehensive test suite covering SUDE functionality, parameter validation, and integration |
| dynamo/tools/utils_reduceDimension.py | Updated to include SUDE as a supported reduction method in the main pipeline |
| dynamo/tools/dimension_reduction.py | Added dedicated run_sude function and updated documentation to mention SUDE support |
| dynamo/tools/init.py | Exposed run_sude function at the package level |
| dynamo/external/sude_py/*.py | Complete SUDE algorithm implementation with supporting mathematical functions |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #729 +/- ##
==========================================
- Coverage 36.56% 30.25% -6.31%
==========================================
Files 183 258 +75
Lines 35352 43407 +8055
==========================================
+ Hits 12925 13131 +206
- Misses 22427 30276 +7849 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…ic weight scheduler for balancing losses
…s with detailed parameter descriptions and usage examples.
…orch`, `pytorch_lightning`, `pandas`
…fix CUDA tensor conversion issues
- Replace sklearn NearestNeighbors with PyTorch-native KNN using torch.cdist for full GPU acceleration
- Remove unnecessary CPU-GPU data transfers by keeping all computations on device
- Fix TypeError: can't convert cuda:0 device type tensor to numpy by adding proper .cpu() calls
- Update forward method to use torch.stack instead of numpy array conversion
- Add device parameter ('cpu'/'gpu') to velocity() function for flexible device selection
- Add GPU memory conflict warning when using multiprocessing with GPU
- Improve tensor-to-scalar conversion with proper device handling
- Renamed _non_para_kernel to _non_para_kernel_t4 for clarity - Added device selection for GPU/CPU in train_vae_nogcn function - Improved folder creation logic with warning for existing directories
…n CI workflows - Added `--exclude=dynamo/external,external` option to flake8 commands in both python-package.yml and python-plain-run-test.yml to prevent linting errors from external directories.
…treamline test suite
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds a new dimension reduction method, SUDE, to the codebase and integrates it into the existing pipeline for dimensionality reduction. The main changes include the addition of the full SUDE algorithm implementation and supporting functions, updating the API to expose SUDE, and modifying documentation to reflect the new option.
SUDE algorithm and supporting functions:
sude.py, including landmark selection, embedding computation, and handling of large datasets.init_pca(PCA initialization),pca(PCA embedding),mds(MDS embedding),pps(landmark sampling),learning_sandlearning_l(manifold learning for landmarks),opt_scale(optimal landmark scaling),clle(constrained locally linear embedding), andplotcluster2(embedding visualization).API and integration updates:
sudefunction at the package level by updating the__init__.pyfile to import it.run_sudeas an available dimension reduction method.reduceDimensionfunction to mention SUDE as a supported reduction method.