Rng improvements #2534
Open
Rng improvements #2534
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ange in GA with new mutation and crossover type
… identified and fixed by Khang where non-objective optimization values were not being returned by the GA.
…s removes the arbitrary restriction requiring DataObjects to include both 'Inputs' and 'Outputs' nodes, despite the situations where that doesn't makes sense.
…e in the _SolutionExport of NSGA-ii that resulted in model inputs and outputs being coupledi incorrectly. This hotfix does NOT correct the same issue with RAVEN's estimation for the 'final' best values.
…ing of optimizer results and the crowding distance calculation for NSGA-ii. Tested with the GA test suite and stress tested with the ZDT1 test in particular.
… hardcoded seed value (e.g. 5489) has been replaced with the default seed value of 'None', which prompts numpy.random to take a high-entropy seed value from the OS (e.g. the system clock). A new subnode was added to <RunInfo> to allow for a globalSeed to be set in RAVEN prior to any code execution, which ensures a user-supplied RNG seed is applied before any RNG calls are made, if desired. Setting this globalSeed value to 5489 was necessary in all test files to ensure backwards compatibility with old gold results.
Equilibrium cycle work june25
…o be calculated incorrectly. (#2) Co-authored-by: Rollins <[email protected]>
…on on the output values prior to calculating the fitness. This had to be implemented separately from the standard RAVEN noramlizeData methodology, as we didn't want to normalize the inputs or return the output values to the user in a normalized format; the normalized values are ONLY needed to estimate the fitness when requested.
…en the solution inputs, constraints, and objectives. The culprit was a dict.update() line that was overwriting the correct values with desynced values. This line was necessary because the self._solutionExport() is not being defined correctly. This will be fixed in a subsequent commit.
… the solutions. The desyncing occurs when the 'populationFitness' local variable (used by single-objective optimization) is stored as the 'self.fitness' attribute of the Algorithm by the survivor selection method. NSGA-II can use the populationFitness local variable just fine, so the 'self.fitness' attribute is superfluous anyway.
… support a reduced input format. Penalty scaling factors are now interpreted as a 2d-array of shape (len(objVar),constraintNum). Function docstrings have been updated accordingly.
…e the way the kwargs dict was being provided to the function.
…solutions to make sure each part of the reproduction process was using the correct values and data for grandparents, parents, and children and that these data were being stored appropriately without overwriting. This in turn fixed the fitness value desyncing issue in NSGA-II as well.
…ividuals from GA are correctly added to and printed with the list of final solutions in the _solutionExport.
…ce redundancy and prevent data from being deleted unnecessarily.
… hardcoded seed value (e.g. 5489) has been replaced with the default seed value of 'None', which prompts numpy.random to take a high-entropy seed value from the OS (e.g. the system clock). A new subnode was added to <RunInfo> to allow for a globalSeed to be set in RAVEN prior to any code execution, which ensures a user-supplied RNG seed is applied before any RNG calls are made, if desired. Setting this globalSeed value to 5489 was necessary in all test files to ensure backwards compatibility with old gold results.
j-bryan
requested changes
Sep 11, 2025
Collaborator
j-bryan
left a comment
j-bryan
left a comment
There was a problem hiding this comment.
The proposed change is much needed for improved reproducibility and random sampling. Only a few minor changes are requested to keep code within the RAVEN code standards and to make the provided global seed conversion script suitable to be run on any machine. Once all tests are passing, this pull request can be approved.
added 2 commits
September 15, 2025 14:32
…ests were updated to have the proper expliciting seeding of the RNG. The unseeded test in testRandomUtils.py was modified to check 5 random floats for any repeats, which could indicate the RNG is failing.
j-bryan
requested changes
Sep 16, 2025
| tests/framework/ROM/TimeSeries/SyntheticHistory/LogARMA/ | ||
| tests/framework/ROM/TimeSeries/SyntheticHistory/VARMA/ | ||
| tests/framework/ROM/TimeSeries/SyntheticHistory/ZeroFilterDiscontinuous/ | ||
| *.xml.bak |
Collaborator
There was a problem hiding this comment.
Make sure you don't commit any *.xml.bak files, but we shouldn't need to have this in the .gitignore file.
…ainties in the fitted coefficients due to the small amount of training data.
…ing inconsistencies on the Linux OS test machines. This will be raised and corrected in an issue.
|
Job Mingw Test on 392a784 : invalidated by @joshua-cogliati-inl restarted civet |
W0lfShAd0w
added a commit
to W0lfShAd0w/raven
that referenced
this pull request
Oct 30, 2025
* The outdated behavior of having randomUtils initialize the RNG with a hardcoded seed value (e.g. 5489) has been replaced with the default seed value of 'None', which prompts numpy.random to take a high-entropy seed value from the OS (e.g. the system clock). A new subnode was added to <RunInfo> to allow for a globalSeed to be set in RAVEN prior to any code execution, which ensures a user-supplied RNG seed is applied before any RNG calls are made, if desired. Setting this globalSeed value to 5489 was necessary in all test files to ensure backwards compatibility with old gold results. * globalSeed parameter of RunInfo now supports 'None' as a valid input. * Added a print statement for when no GlobalSeed is provided. * Minor changes made to address comments in PR idaholab#2534. Several tests were updated to have the proper expliciting seeding of the RNG. The unseeded test in testRandomUtils.py was modified to check 5 random floats for any repeats, which could indicate the RNG is failing. * global seed added to more tests to ensure consistency with golds. * Minor change to clarify output messages from globalSeed check. * Deprecate Rattlesnake and Mammoth (idaholab#2519) * remove Rattlesnake, Mammoth, and Instant tests * remove Rattlesnake and Mammoth codeinterfaces * removing from CodeInterface factory * removing Mammoth and Rattlesnake references in the docs * Added globalSeed parameter to test input file for backwards compatability. * Modified the tolerance on the multiYearDWT test to account for uncertainties in the fitted coefficients due to the small amount of training data. * Increased tolerances further for multiYearDWT test to get around fitting inconsistencies on the Linux OS test machines. This will be raised and corrected in an issue. * Attempt at addressing the possible intermittent test error on Fedora machine. --------- Co-authored-by: Rollins <[email protected]> Co-authored-by: Rollins <[email protected]> Co-authored-by: Gabriel J. Soto Gonzalez <[email protected]> Co-authored-by: Rollins <[email protected]> Co-authored-by: rollnk <[email protected]>
W0lfShAd0w
added a commit
to W0lfShAd0w/raven
that referenced
this pull request
Oct 30, 2025
* Rng improvements (#3) * The outdated behavior of having randomUtils initialize the RNG with a hardcoded seed value (e.g. 5489) has been replaced with the default seed value of 'None', which prompts numpy.random to take a high-entropy seed value from the OS (e.g. the system clock). A new subnode was added to <RunInfo> to allow for a globalSeed to be set in RAVEN prior to any code execution, which ensures a user-supplied RNG seed is applied before any RNG calls are made, if desired. Setting this globalSeed value to 5489 was necessary in all test files to ensure backwards compatibility with old gold results. * globalSeed parameter of RunInfo now supports 'None' as a valid input. * Added a print statement for when no GlobalSeed is provided. * Minor changes made to address comments in PR idaholab#2534. Several tests were updated to have the proper expliciting seeding of the RNG. The unseeded test in testRandomUtils.py was modified to check 5 random floats for any repeats, which could indicate the RNG is failing. * global seed added to more tests to ensure consistency with golds. * Minor change to clarify output messages from globalSeed check. * Deprecate Rattlesnake and Mammoth (idaholab#2519) * remove Rattlesnake, Mammoth, and Instant tests * remove Rattlesnake and Mammoth codeinterfaces * removing from CodeInterface factory * removing Mammoth and Rattlesnake references in the docs * Added globalSeed parameter to test input file for backwards compatability. * Modified the tolerance on the multiYearDWT test to account for uncertainties in the fitted coefficients due to the small amount of training data. * Increased tolerances further for multiYearDWT test to get around fitting inconsistencies on the Linux OS test machines. This will be raised and corrected in an issue. * Attempt at addressing the possible intermittent test error on Fedora machine. --------- Co-authored-by: Rollins <[email protected]> Co-authored-by: Rollins <[email protected]> Co-authored-by: Gabriel J. Soto Gonzalez <[email protected]> Co-authored-by: Rollins <[email protected]> Co-authored-by: rollnk <[email protected]> * Optimizer data storage fixes (#5) * Minor bug fixes and formatting improvements. Fixed the bug previously identified and fixed by Khang where non-objective optimization values were not being returned by the GA. * Applied hotfix from Khang Nguyen that corrects a synchronization issue in the _SolutionExport of NSGA-ii that resulted in model inputs and outputs being coupledi incorrectly. This hotfix does NOT correct the same issue with RAVEN's estimation for the 'final' best values. * Further improvements from Khang and Mohammad Abdo to improve the syncing of optimizer results and the crowding distance calculation for NSGA-ii. Tested with the GA test suite and stress tested with the ZDT1 test in particular. * Temporary fix made to correct desyncing issues in the Optimizer between the solution inputs, constraints, and objectives. The culprit was a dict.update() line that was overwriting the correct values with desynced values. This line was necessary because the self._solutionExport() is not being defined correctly. This will be fixed in a subsequent commit. * Fixed a bug in NSGA-II causing the fitness values to be desynced from the solutions. The desyncing occurs when the 'populationFitness' local variable (used by single-objective optimization) is stored as the 'self.fitness' attribute of the Algorithm by the survivor selection method. NSGA-II can use the populationFitness local variable just fine, so the 'self.fitness' attribute is superfluous anyway. * Overhauled the method by which NSGA-II was storing data/tracking for solutions to make sure each part of the reproduction process was using the correct values and data for grandparents, parents, and children and that these data were being stored appropriately without overwriting. This in turn fixed the fitness value desyncing issue in NSGA-II as well. * removed tab character. * removed trailing whitespace from comment line. * Commented out a line that is an artifact from cherry-picking from a different branch. This needs to be re-added in a future merge. * Fixed new variable name to match the camelCase convention used in RAVEN. Also, fixed a bug where unneeded multiobjective variables were expected but not initialized in single objective GA. --------- Co-authored-by: Rollins <[email protected]> Co-authored-by: Rollins <[email protected]> Co-authored-by: rollnk <[email protected]> --------- Co-authored-by: Rollins <[email protected]> Co-authored-by: Rollins <[email protected]> Co-authored-by: Gabriel J. Soto Gonzalez <[email protected]> Co-authored-by: Rollins <[email protected]> Co-authored-by: rollnk <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)
The outdated behavior of having randomUtils initialize the RNG with a hardcoded seed value (e.g. 5489) has been replaced with the default seed value of 'None', which prompts numpy.random to use high-entropy initial state. A new input subnode was added to to allow for a globalSeed to be set in RAVEN prior to any code execution, which ensures a user-supplied RNG seed is applied before any RNG calls are made, if desired. Setting this globalSeed value to 5489 was necessary in all test files to ensure backwards compatibility with old gold results.
What are the significant changes in functionality due to this change request?
For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.
<internalParallel>to True.raven/tests/framework/user_guideandraven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.