Added test case to verify capsule upgradation with hammer installed #19933

nacoool · 2025-10-10T10:05:40Z

Problem Statement

There are no automated checks to verify Capsule Hammer functionality before and after an upgrade. Manual validation is time-consuming and prone to human error.

Solution

Added a new test class TestCapsuleHammer that validates Capsule Hammer in two phases:

Pre-upgrade: Confirms capsule configuration and baseline functionality before upgrade.
Post-upgrade: Runs dependent validation after upgrade to ensure functionality remains intact.

Related Issues

sourcery-ai

Hey there - I've reviewed your changes - here's some feedback:

Instead of hardcoding “satellite-capsule-6.19” in the post-upgrade check, parameterize the expected capsule version from settings or fixtures so it doesn’t break on version bumps.
Consider extracting repeated command execution and assertion logic (e.g. yum repo creation, package installation checks) into helper functions or fixtures to reduce duplication and improve readability.
Double-check the pytest.mark.post_upgrade(depend_on=…) usage to ensure it correctly references the pre-upgrade test (string or marker) so the dependency is actually enforced.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- Instead of hardcoding “satellite-capsule-6.19” in the post-upgrade check, parameterize the expected capsule version from settings or fixtures so it doesn’t break on version bumps.
- Consider extracting repeated command execution and assertion logic (e.g. yum repo creation, package installation checks) into helper functions or fixtures to reduce duplication and improve readability.
- Double-check the pytest.mark.post_upgrade(depend_on=…) usage to ensure it correctly references the pre-upgrade test (string or marker) so the dependency is actually enforced.

## Individual Comments

### Comment 1
<location> `tests/upgrades/test_capsulecontent.py:296-297` </location>
<code_context>
+        assert result.status == 0
+        result = module_capsule_configured.execute('yum install rubygem-hammer_cli_katello -y')
+        assert result.status == 0
+        result = module_capsule_configured.execute('rpm -qa | grep -i "hammer"')
+        assert "hammer_cli_katello" in result.stdout
+        assert result.status == 0
+
</code_context>

<issue_to_address>
**suggestion (testing):** Suggestion to verify hammer CLI functionality, not just installation.

Consider adding a test that runs a basic hammer CLI command to ensure it works as expected after installation.

```suggestion
        result = module_capsule_configured.execute('yum install rubygem-hammer_cli_katello -y')
        assert result.status == 0
        # Verify hammer CLI is functional
        result = module_capsule_configured.execute('hammer --version')
        assert result.status == 0
        assert "Hammer CLI" in result.stdout or "hammer" in result.stdout.lower()
```
</issue_to_address>

### Comment 2
<location> `tests/upgrades/test_capsulecontent.py:317-318` </location>
<code_context>
+            1. The capsule should be upgraded.
+            2. The hammer should be installed on Capsule.
+        """
+        result = module_capsule_configured.execute('rpm -qa | grep -i "capsule"')
+        assert "satellite-capsule-6.19" in result.stdout
+        assert result.status == 0
+        result = module_capsule_configured.execute('rpm -qa | grep -i "hammer"')
</code_context>

<issue_to_address>
**question (testing):** Question about hardcoded version check for capsule upgrade.

If the expected version changes in future releases, this test may fail. Please consider parameterizing the version or using a more flexible check.
</issue_to_address>

### Comment 3
<location> `tests/upgrades/test_capsulecontent.py:302-303` </location>
<code_context>
+        assert "hammer_cli_katello" in result.stdout
+        assert result.status == 0
+
+    @pytest.mark.post_upgrade(depend_on=test_pre_upgrade_check_capsule_hammer)
+    def test_post_upgrade_check_capsule_hammer(self, module_capsule_configured):
+        """Post-upgrade scenario that verifies if the hammer is installed on Capsule
+        in the post-upgrade.
</code_context>

<issue_to_address>
**suggestion (testing):** Suggestion to add validation for repo sync after upgrade.

Please add a check to confirm the utils repository and its RPMs remain available and properly synced after the upgrade.

Suggested implementation:

```python
        result = module_capsule_configured.execute('yum install rubygem-hammer_cli_katello -y')
        assert result.status == 0
        result = module_capsule_configured.execute('rpm -qa | grep -i "hammer"')
        assert "hammer_cli_katello" in result.stdout
        assert result.status == 0

        # Validate utils repository and RPMs remain available and properly synced after upgrade
        result = module_capsule_configured.execute('yum repolist all')
        assert "Utils Repository" in result.stdout
        assert result.status == 0

        # Check for a known RPM from the utils repo (replace 'some-utils-rpm' with actual RPM name if needed)
        result = module_capsule_configured.execute('yum list available | grep -i "some-utils-rpm"')
        assert result.status == 0
        assert "some-utils-rpm" in result.stdout

        # Try installing the RPM to confirm availability
        result = module_capsule_configured.execute('yum install some-utils-rpm -y')
        assert result.status == 0

    @pytest.mark.post_upgrade(depend_on=test_pre_upgrade_check_capsule_hammer)
    def test_post_upgrade_check_capsule_hammer(self, module_capsule_configured):

```

- Replace "some-utils-rpm" with the actual RPM name from the utils repository that should be validated.
- If the repository name in `yum repolist all` output differs, adjust "Utils Repository" accordingly.
- If you have a helper for repo sync status, consider using it for more robust validation.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

sourcery-ai · 2025-10-10T10:06:22Z

tests/upgrades/test_capsulecontent.py

+    @pytest.mark.post_upgrade(depend_on=test_pre_upgrade_check_capsule_hammer)
+    def test_post_upgrade_check_capsule_hammer(self, module_capsule_configured):


suggestion (testing): Suggestion to add validation for repo sync after upgrade.

Please add a check to confirm the utils repository and its RPMs remain available and properly synced after the upgrade.

Suggested implementation:

result = module_capsule_configured.execute('yum install rubygem-hammer_cli_katello -y') assert result.status == 0 result = module_capsule_configured.execute('rpm -qa | grep -i "hammer"') assert "hammer_cli_katello" in result.stdout assert result.status == 0 # Validate utils repository and RPMs remain available and properly synced after upgrade result = module_capsule_configured.execute('yum repolist all') assert "Utils Repository" in result.stdout assert result.status == 0 # Check for a known RPM from the utils repo (replace 'some-utils-rpm' with actual RPM name if needed) result = module_capsule_configured.execute('yum list available | grep -i "some-utils-rpm"') assert result.status == 0 assert "some-utils-rpm" in result.stdout # Try installing the RPM to confirm availability result = module_capsule_configured.execute('yum install some-utils-rpm -y') assert result.status == 0 @pytest.mark.post_upgrade(depend_on=test_pre_upgrade_check_capsule_hammer) def test_post_upgrade_check_capsule_hammer(self, module_capsule_configured):

Replace "some-utils-rpm" with the actual RPM name from the utils repository that should be validated.

If the repository name in yum repolist all output differs, adjust "Utils Repository" accordingly.

If you have a helper for repo sync status, consider using it for more robust validation.

vsedmik

I'm afraid this test needs to go into tests/new_upgrades and be updated accordingly.

The legacy tests/upgrades are not to be run in CI.

nacoool · 2025-10-10T10:20:19Z

trigger: test-robottelo
pytest: tests/upgrades/test_capsulecontent.py -k TestCapsuleHammer
env:
ROBOTTELO_server__deploy_arguments__deploy_sat_version: '6.18'

nacoool · 2025-10-10T10:46:47Z

I'm afraid this test needs to go into tests/new_upgrades and be updated accordingly.

The legacy tests/upgrades are not to be run in CI.

I have added to new_upgrades now.

nacoool · 2025-10-10T10:51:03Z

trigger: test-robottelo
pytest: tests/upgrades/test_capsulecontent.py -k "TestCapsuleHammer"
env:
ROBOTTELO_server__deploy_arguments__deploy_sat_version: '6.18'

nacoool · 2025-10-10T10:58:56Z

trigger: test-robottelo
pytest: tests/new_upgrades/test_capsulecontent.py::TestCapsuleHammer
env:
ROBOTTELO_server__deploy_arguments__deploy_sat_version: '6.18'

nacoool · 2025-10-10T11:07:13Z

trigger: test-robottelo
pytest: tests/new_upgrades/test_capsulecontent.py::TestCapsuleHammer
env:
    ROBOTTELO_server__deploy_workflows__product: 'deploy-satellite-upgrade'
    ROBOTTELO_server__deploy_arguments__deploy_rhel_version: '9'
    ROBOTTELO_server__deploy_arguments__deploy_sat_version: '6.18'
    ROBOTTELO_server__deploy_arguments__deploy_snap_version: 'ystream'
    BROKER_AnsibleTower__inventory: 'osp-rhos01-satellite-upgrade'

Satellite-QE · 2025-10-10T11:21:16Z

PRT Result

Build Number: 13118
Build Status: UNSTABLE
PRT Comment: pytest tests/new_upgrades/test_capsulecontent.py::TestCapsuleHammer --external-logging
Test Result : ======================== 4 warnings, 2 errors in 38.32s ========================

nacoool · 2025-10-15T08:46:21Z

I'm afraid this test needs to go into tests/new_upgrades and be updated accordingly.

The legacy tests/upgrades are not to be run in CI.

I have added it to tests/foreman/maintain/test_upgrade.py with respect to component mentioned in the bug.

vsedmik

Proposed a couple of nit-picks, otherwise looks good.

vsedmik · 2025-10-15T10:06:38Z