Skip to content

Conversation

@krcmarik
Copy link
Contributor

@krcmarik krcmarik commented Jan 12, 2026

The VMs created on destination OCP cluster by migration process are deleted at the end of the test session once all tests/migration plans are executed and not at the end of the each finished migration plan. If more tests with warm migration type or target power state - on are sequentially executed within one session, a smaller OCP cluster may quickly end up without sufficient resources:
0/6 nodes are available: 3 Insufficient cpu, 3 Insufficient memory.

To mitigate the problem we may do:

  • Delete VMs immediately after each test instead of deferring to session end
  • Remove Pod tracking as Kubernetes garbage collects them via ownerReferences
  • Remove redundant VM cleanup loop from session teardown (delete_all_vms serves as safety net)

(Or we can alternatively make sure that in the cleanup phase of Plan fixture we stop all the running VMs created withint the Plan execution)

Summary by CodeRabbit

  • Tests
    • Test teardown now deletes virtual machines immediately after plans, with improved error handling and logging.
    • Pods are no longer collected for teardown and rely on Kubernetes garbage collection.
    • Removed deferred teardown tracking and duplicate VM cleanup logic to simplify resource cleanup.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Jan 12, 2026

Warning

Rate limit exceeded

@krcmarik has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 9 minutes and 15 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between c223a54 and 100d88c.

📒 Files selected for processing (1)
  • conftest.py

Walkthrough

Session-level VM/Pod teardown was removed and VM deletion moved into the test fixture: after yielding the plan, the fixture now immediately attempts to instantiate and delete each plan["virtual_machines"] with try/except and logging; Pods are no longer collected for teardown (handled by Kubernetes GC).

Changes

Cohort / File(s) Summary
Fixture-level cleanup
conftest.py
After plan is yielded, iterate plan["virtual_machines"], instantiate VirtualMachine objects in the test namespace, call clean_up(wait=True) for each, and wrap deletion in try/except with logging. Removed populating fixture_store teardown entries for VMs and Pods; Pods are left to Kubernetes GC via ownerReferences.
Session teardown simplification
utilities/pytest_utils.py
Removed VirtualMachine import and removed retrieval/cleanup logic for VirtualMachine entries from session-level session_teardown_resources (no more end-of-session VM cleanup loop or associated error handling).
Repository metadata
manifest_file, requirements.txt, pyproject.toml, setup.py
No functional changes to code; listed as context in diff.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Implement immediate VM cleanup in plan fixture' directly summarizes the main change—moving VM deletion from session end to after each plan execution.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@redhat-qe-bot1
Copy link

Report bugs in Issues

Welcome! 🎉

This pull request will be automatically processed with the following features:

🔄 Automatic Actions

  • Reviewer Assignment: Reviewers are automatically assigned based on the OWNERS file in the repository root
  • Size Labeling: PR size labels (XS, S, M, L, XL, XXL) are automatically applied based on changes
  • Issue Creation: Disabled for this repository
  • Pre-commit Checks: pre-commit runs automatically if .pre-commit-config.yaml exists
  • Branch Labeling: Branch-specific labels are applied to track the target branch
  • Auto-verification: Auto-verified users have their PRs automatically marked as verified

📋 Available Commands

PR Status Management

  • /wip - Mark PR as work in progress (adds WIP: prefix to title)
  • /wip cancel - Remove work in progress status
  • /hold - Block PR merging (approvers only)
  • /hold cancel - Unblock PR merging
  • /verified - Mark PR as verified
  • /verified cancel - Remove verification status
  • /reprocess - Trigger complete PR workflow reprocessing (useful if webhook failed or configuration changed)

Review & Approval

  • /lgtm - Approve changes (looks good to me)
  • /approve - Approve PR (approvers only)
  • /automerge - Enable automatic merging when all requirements are met (maintainers and approvers only)
  • /assign-reviewers - Assign reviewers based on OWNERS file
  • /assign-reviewer @username - Assign specific reviewer
  • /check-can-merge - Check if PR meets merge requirements

Testing & Validation

  • /retest tox - Run Python test suite with tox
  • /retest build-container - Rebuild and test container image
  • /retest pre-commit - Run pre-commit hooks and checks
  • /retest all - Run all available tests

Container Operations

  • /build-and-push-container - Build and push container image (tagged with PR number)
    • Supports additional build arguments: /build-and-push-container --build-arg KEY=value

Cherry-pick Operations

  • /cherry-pick <branch> - Schedule cherry-pick to target branch when PR is merged
    • Multiple branches: /cherry-pick branch1 branch2 branch3

Label Management

  • /<label-name> - Add a label to the PR
  • /<label-name> cancel - Remove a label from the PR

✅ Merge Requirements

This PR will be automatically approved when the following conditions are met:

  1. Approval: /approve from at least one approver
  2. LGTM Count: Minimum 0 /lgtm from reviewers
  3. Status Checks: All required status checks must pass
  4. No Blockers: No WIP, hold, or conflict labels
  5. Verified: PR must be marked as verified (if verification is enabled)

📊 Review Process

Approvers and Reviewers

Approvers:

  • myakove

Reviewers:

  • AmenB
  • Chenli-Hu
  • krcmarik
  • myakove
  • rnetser
Available Labels
  • hold
  • verified
  • wip
  • lgtm
  • approve
  • automerge

💡 Tips

  • WIP Status: Use /wip when your PR is not ready for review
  • Verification: The verified label is automatically removed on each new commit
  • Cherry-picking: Cherry-pick labels are processed when the PR is merged
  • Container Builds: Container images are automatically tagged with the PR number
  • Permission Levels: Some commands require approver permissions
  • Auto-verified Users: Certain users have automatic verification and merge privileges

For more information, please refer to the project documentation or contact the maintainers.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In @conftest.py:
- Around line 739-740: Replace the LOGGER.error call in the VM cleanup except
block so that the full traceback is preserved: change the except block that
catches Exception (current code using LOGGER.error(f"Failed to delete VM
{vm['name']}: {exc}")) to call LOGGER.exception with %-style formatting (e.g.,
LOGGER.exception("Failed to delete VM %s", vm["name"])) and remove the unused
exc variable; this ensures the stack trace is logged and avoids f-string lint
warnings.
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2063edf and 66a9ff7.

📒 Files selected for processing (2)
  • conftest.py
  • utilities/pytest_utils.py
💤 Files with no reviewable changes (1)
  • utilities/pytest_utils.py
🧰 Additional context used
🪛 Ruff (0.14.10)
conftest.py

737-737: Logging statement uses f-string

(G004)


739-739: Do not catch blind exception: Exception

(BLE001)


740-740: Use logging.exception instead of logging.error

Replace with exception

(TRY400)


740-740: Logging statement uses f-string

(G004)

🔇 Additional comments (1)
conftest.py (1)

727-738: LGTM! Immediate VM cleanup aligns with PR objectives

The cleanup logic correctly:

  • Targets destination VMs in target_namespace (not source VMs)
  • Uses exists check to avoid NotFoundError on already-deleted VMs
  • Employs wait=True for synchronous deletion, preventing resource contention between sequential tests
  • Handles the case where vm["name"] was updated to the migrated name on line 711

This directly addresses the resource exhaustion issue on smaller clusters.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In @conftest.py:
- Around line 739-740: Remove the unused exception variable by changing the
except clause from "except Exception as exc:" to "except Exception:" around the
block that calls LOGGER.exception("Failed to delete VM %s", vm["name"]);
LOGGER.exception already logs the current exception context so drop the "as exc"
binding and leave the LOGGER.exception call unchanged.
- Line 737: Replace the eager f-string in the LOGGER.info call with lazy logging
interpolation: change the LOGGER.info invocation that currently uses f"Deleting
VM {vm_obj.name} after test" to use a format string and argument(s) so the
message is constructed only if the log level is enabled (i.e.,
LOGGER.info("Deleting VM %s after test", vm_obj.name)); update the specific
LOGGER.info call that references vm_obj.name accordingly.
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 66a9ff7 and 4a99f8a.

📒 Files selected for processing (1)
  • conftest.py
🧰 Additional context used
🪛 Ruff (0.14.10)
conftest.py

737-737: Logging statement uses f-string

(G004)


739-739: Local variable exc is assigned to but never used

Remove assignment to unused variable exc

(F841)

🔇 Additional comments (2)
conftest.py (2)

727-729: LGTM on the cleanup strategy.

Moving VM cleanup from session-level to function-level addresses the resource exhaustion issue on smaller clusters. The comment about Pods being garbage collected via ownerReferences is accurate — Kubernetes will handle pod cleanup automatically.


730-740: No issues found. The vm["name"] mutation at line 711 is intentional and correct. The source_provider.vm_dict() method returns the actual cloned/renamed VM name in source_vm_details["name"]. Updating vm["name"] with this value ensures cleanup uses the correct actual VM name in the target namespace, which is the proper design. The updated name is consistently used throughout (line 717 waits for the updated name, line 733 cleans up the updated name).

- Delete VMs immediately after each test instead of deferring to session end
- Remove Pod tracking as Kubernetes garbage collects them via ownerReferences
- Remove redundant VM cleanup loop from session teardown (delete_all_vms serves as safety net)
- Prevents resource accumulation during long test runs
@krcmarik
Copy link
Contributor Author

@coderabbitai review

@coderabbitai
Copy link

coderabbitai bot commented Jan 12, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@krcmarik
Copy link
Contributor Author

@coderabbitai full review

@coderabbitai
Copy link

coderabbitai bot commented Jan 12, 2026

✅ Actions performed

Full review triggered.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In @conftest.py:
- Around line 727-740: The LOGGER.info call uses an f-string while other logs
use lazy % formatting; change the info call inside the VM cleanup loop to use
lazy formatting by replacing the f-string with LOGGER.info("Deleting VM %s after
test", vm_obj.name) so logging is consistent with LOGGER.exception("Failed to
delete VM %s", vm["name"]) and avoids eager interpolation.
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2063edf and c223a54.

📒 Files selected for processing (2)
  • conftest.py
  • utilities/pytest_utils.py
💤 Files with no reviewable changes (1)
  • utilities/pytest_utils.py
🧰 Additional context used
🪛 Ruff (0.14.10)
conftest.py

737-737: Logging statement uses f-string

(G004)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: can-be-merged
  • GitHub Check: tox
  • GitHub Check: pre-commit
  • GitHub Check: build-container
🔇 Additional comments (1)
conftest.py (1)

729-740: Immediate per-test VM cleanup is a solid improvement.

This change directly addresses the resource exhaustion issue described in the PR objectives. Deleting VMs immediately after each test rather than deferring to session end prevents OCP clusters from running out of CPU/memory when running sequential warm migrations or migrations with target power state "on".

The assumption that Kubernetes will garbage-collect pods via ownerReferences is correct - VMs own their virt-launcher pods, so pod cleanup is automatic.

"module": pod.__module__,
})
try:
vm_obj = VirtualMachine(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change defeats the whole process of having a functional environment for debugging if someone needs it.
If the issue is resources then I prefer to stop the VMs and not delete them here.
And do not remove the exsiting code, we store them also in order to get leftovers for the current run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants