Skip to content

Add new operations to Random operations test #12213

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jul 22, 2025
Merged

Conversation

a-masterov
Copy link
Contributor

@a-masterov a-masterov commented Jun 12, 2025

Problem

We did not test some Public API calls, such as using a timestamp to create a branch, reset_to_parent.

Summary of changes

Tests now include some other operations: reset_to_parent, a branch creation from any time in the past, etc.
Currently, the API calls are only exposed; the semantics are not verified.

@a-masterov a-masterov added the run-no-ci Don't run any CI for this PR. label Jun 12, 2025
@a-masterov a-masterov changed the base branch from main to amasterov/workaround-branch-not-found-problem July 9, 2025 12:55
Base automatically changed from amasterov/workaround-branch-not-found-problem to main July 9, 2025 15:30
Alexey Masterov added 2 commits July 9, 2025 18:59
# Conflicts:
#	test_runner/random_ops/test_random_ops.py
@a-masterov a-masterov force-pushed the amasterov/random-ops-add branch from 0fe07d7 to 00328e8 Compare July 15, 2025 10:06
@a-masterov a-masterov force-pushed the amasterov/random-ops-add branch 2 times, most recently from 55c8232 to e97c1d2 Compare July 15, 2025 12:49
@a-masterov a-masterov removed the run-no-ci Don't run any CI for this PR. label Jul 15, 2025
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: Add new operations to Random operations test

Approved - Great expansion of test coverage!

This PR significantly enhances the Random operations test by adding comprehensive snapshot functionality and reset-to-parent operations. The implementation is well-structured and follows existing patterns.

Strengths:

  • Comprehensive snapshot support: Full lifecycle management (create, restore, delete) with proper API integration
  • Enhanced timing logic: The method properly handles timing constraints to avoid invalid timestamps
  • Proper benchmark coordination: Correctly terminates and restarts pgbench processes during disruptive operations
  • Good error handling: Proper limit checking and graceful fallbacks when operations can't be performed
  • Weighted operation distribution: Thoughtful probability weights for new operations (snapshots have low weights for gradual introduction)

Suggestions for improvement:

  1. 🔧 Remove debug log (Line 122):

    # XXX for debug only, remove before merge
    log.info("%s", branch["branch"])

    This debug log should be removed before merging.

  2. 📝 Consider adding docstrings for new methods like reset_to_parent() and create_snapshot() to match the existing documentation style.

  3. 🧪 Test edge cases: Consider testing behavior when:

    • No snapshots exist for restore/delete operations
    • Branch limits are reached during snapshot restoration
    • Timing constraints prevent valid random time generation

Technical Notes:

  • The new NeonSnapshot class follows the same pattern as NeonBranch and NeonEndpoint
  • API client additions in neon_api.py are well-structured with proper type hints
  • The do_action() function cleanly handles the 5 new operation types
  • Benchmark restart logic during reset operations looks robust

Testing Recommendation:

Run with a smaller operation count (e.g., 50) initially to validate the new snapshot and reset operations work correctly under various timing conditions.

Great work expanding the test coverage! 🚀

Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: Add new operations to Random operations test

Approved - Great expansion of test coverage!

This PR significantly enhances the Random operations test by adding comprehensive snapshot functionality and reset-to-parent operations. The implementation is well-structured and follows existing patterns.

Strengths:

  • Comprehensive snapshot support: Full lifecycle management (create, restore, delete) with proper API integration
  • Enhanced timing logic: The random_time() method properly handles timing constraints to avoid invalid timestamps
  • Proper benchmark coordination: Correctly terminates and restarts pgbench processes during disruptive operations
  • Good error handling: Proper limit checking and graceful fallbacks when operations can't be performed
  • Weighted operation distribution: Thoughtful probability weights for new operations (snapshots have low weights for gradual introduction)

Suggestions for improvement:

  1. 🔧 Remove debug log (Line 122):

    # XXX for debug only, remove before merge
    log.info("%s", branch["branch"])

    This debug log should be removed before merging.

  2. 📝 Consider adding docstrings for new methods like reset_to_parent() and create_snapshot() to match the existing documentation style.

  3. 🧪 Test edge cases: Consider testing behavior when:

    • No snapshots exist for restore/delete operations
    • Branch limits are reached during snapshot restoration
    • Timing constraints prevent valid random time generation

Technical Notes:

  • The new NeonSnapshot class follows the same pattern as NeonBranch and NeonEndpoint
  • API client additions in neon_api.py are well-structured with proper type hints
  • The do_action() function cleanly handles the 5 new operation types
  • Benchmark restart logic during reset operations looks robust

Testing Recommendation:

Run with a smaller operation count (e.g., 50) initially to validate the new snapshot and reset operations work correctly under various timing conditions.

Great work expanding the test coverage! 🚀

Copy link

github-actions bot commented Jul 15, 2025

8910 tests run: 8245 passed, 0 failed, 665 skipped (full report)


Flaky tests (8)

Postgres 17

Postgres 16

Postgres 15

Postgres 14

Code coverage* (full report)

  • functions: 34.8% (8796 of 25281 functions)
  • lines: 46.0% (71105 of 154711 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
4be46b1 at 2025-07-22T17:53:49.494Z :recycle:

@a-masterov a-masterov marked this pull request as ready for review July 15, 2025 17:44
@a-masterov a-masterov requested review from bayandin, Bodobolero and a team July 16, 2025 10:20
@a-masterov a-masterov marked this pull request as draft July 17, 2025 12:11
@a-masterov a-masterov force-pushed the amasterov/random-ops-add branch 3 times, most recently from cdc2ea1 to 1443ba6 Compare July 21, 2025 08:44
@a-masterov a-masterov marked this pull request as ready for review July 21, 2025 08:49
Copy link
Contributor

@Bodobolero Bodobolero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@a-masterov a-masterov enabled auto-merge July 22, 2025 16:31
@a-masterov a-masterov added this pull request to the merge queue Jul 22, 2025
Merged via the queue into main with commit b384490 Jul 22, 2025
97 checks passed
@a-masterov a-masterov deleted the amasterov/random-ops-add branch July 22, 2025 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants