Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1067 improve abm tests #1141

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

1067 improve abm tests #1141

wants to merge 6 commits into from

Conversation

khoanguyen-dev
Copy link
Member

@khoanguyen-dev khoanguyen-dev commented Oct 31, 2024

Changes and Information

Please briefly list the changes (main added features, changed items, or corrected bugs) made:

  • Split up TestWorld.evolveMigration.
  • Add fuzzing or more test cases to TestPerson and TestLocation.
  • Add all checks for different VirusVariants in check_constraints() of parameters.h.

If need be, add additional information and what the reviewer should look out for in particular:

Benchmark results:

  • Current branch:
    Run on (8 X 24.1214 MHz CPU s)
    CPU Caches:
    L1 Data 64 KiB (x8)
    L1 Instruction 128 KiB (x8)
    L2 Unified 4096 KiB (x2)
    Load Average: 2.14, 2.46, 2.84

Benchmark Time CPU Iterations

abm_benchmark/abm_benchmark_50k 1300 ms 1294 ms 1
abm_benchmark/abm_benchmark_100k 2599 ms 2586 ms 1
abm_benchmark/abm_benchmark_200k 5220 ms 5187 ms 1

  • Main branch:
    Run on (8 X 24.0948 MHz CPU s)
    CPU Caches:
    L1 Data 64 KiB (x8)
    L1 Instruction 128 KiB (x8)
    L2 Unified 4096 KiB (x2)
    Load Average: 3.26, 3.54, 3.26

Benchmark Time CPU Iterations

abm_benchmark/abm_benchmark_50k 1311 ms 1306 ms 1
abm_benchmark/abm_benchmark_100k 2611 ms 2607 ms 1
abm_benchmark/abm_benchmark_200k 5242 ms 5225 ms 1

Merge Request - Guideline Checklist

Please check our git workflow. Use the draft feature if the Pull Request is not yet ready to review.

Checks by code author

  • Every addressed issue is linked (use the "Closes #ISSUE" keyword below)
  • New code adheres to coding guidelines
  • No large data files have been added (files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
  • Tests are added for new functionality and a local test run was successful (with and without OpenMP)
  • Appropriate documentation for new functionality has been added (Doxygen in the code and Markdown files if necessary)
  • Proper attention to licenses, especially no new third-party software with conflicting license has been added
  • (For ABM development) Checked benchmark results and ran and posted a local test above from before and after development to ensure performance is monitored.

Checks by code reviewer(s)

  • Corresponding issue(s) is/are linked and addressed
  • Code is clean of development artifacts (no deactivated or commented code lines, no debugging printouts, etc.)
  • Appropriate unit tests have been added, CI passes, code coverage and performance is acceptable (did not decrease)
  • No large data files added in the whole history of commits(files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
  • On merge, add 2-5 lines with the changes (main added features, changed items, or corrected bugs) to the merge-commit-message. This can be taken from the briefly-list-the-changes above (best case) or the separate commit messages (worst case).

@khoanguyen-dev khoanguyen-dev linked an issue Oct 31, 2024 that may be closed by this pull request
2 tasks
Copy link

codecov bot commented Oct 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.59%. Comparing base (86eb344) to head (6b9ff47).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1141   +/-   ##
=======================================
  Coverage   96.59%   96.59%           
=======================================
  Files         137      137           
  Lines       11057    11063    +6     
=======================================
+ Hits        10680    10686    +6     
  Misses        377      377           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@reneSchm reneSchm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes to parameters look fine, only the loop got changed.
The additional comments in and on the test are really nice, especially on the more complicated ones. Though I am unsure whether the doxygen format should be used on a TEST makro over a regular comment.
This definitely improves the tests, but I will keep the linked issue open for now.

Comment on lines +640 to +643
for (auto age_group = AgeGroup(0); age_group < AgeGroup(m_num_groups); ++age_group) {
for (std::uint32_t variant_count = 0; variant_count < static_cast<std::uint32_t>(VirusVariant::Count); ++variant_count) {

auto virus_variant = static_cast<mio::abm::VirusVariant>(variant_count);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With a patch to make_index_range in index_range.h, this becomes

Suggested change
for (auto age_group = AgeGroup(0); age_group < AgeGroup(m_num_groups); ++age_group) {
for (std::uint32_t variant_count = 0; variant_count < static_cast<std::uint32_t>(VirusVariant::Count); ++variant_count) {
auto virus_variant = static_cast<mio::abm::VirusVariant>(variant_count);
for (auto age_group : make_index_range(AgeGroup{m_num_groups})) {
for (auto virus_variant : make_index_range(Index<VirusVariant>{VirusVariant::Count})) {

Try to replace make_index_range with the following, so it can work with AgeGroup (otherwise Index<AgeGroup> is required):

/**
 * @brief Construct a range that can be used to iterate over all MultiIndices in the given dimensions.
 * The range spans over [0, d) for each category in the MultiIndex, where d is that category's value in dimensions.
 * @param[in] dimensions A MultiIndex that contains the dimension for each category.
 * @tparam Categories All categories of the given MultiIndex.
 * @return An iterable range over the given dimensions.
 */
template <class... Categories>
IndexRange<Index<Categories...>> make_index_range(const Index<Categories...>& dimensions)
{
    return IndexRange<Index<Categories...>>(dimensions);
}

Comment on lines +103 to +108
// Fuzzing: assign random valid LocationId values and verify correctness.
for (int i = 0; i < 100; ++i) {
auto random_id = mio::UniformIntDistribution<int>::get_instance()(rng, 0, 1000);
person.set_assigned_location(mio::abm::LocationType::Work, mio::abm::LocationId(random_id));
EXPECT_EQ(person.get_assigned_location(mio::abm::LocationType::Work), mio::abm::LocationId(random_id));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be further improved, for one by making use of the new "RandomNumberTest" (maybe with a random_integer function) to handle the rng and seed logging. And we can fuzz the Location type in addition to the id.

EXPECT_EQ(person.get_assigned_location(mio::abm::LocationType::Work), mio::abm::LocationId(random_id));
}

// Boundary test cases: test with boundary LocationIds.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It says boundary edge cases, but above the upper bound is only 1000, not max. I think the wording here should change to fix this, or maybe do use max() as upper bound for the rng

@reneSchm reneSchm removed a link to an issue Nov 12, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants