Skip to content

Fix E2E test reliability and improve CI/CD workflow #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jun 15, 2025

Conversation

JimboSchneider
Copy link
Owner

Summary

  • Temporarily disable flaky E2E tests in PR build workflow
  • Fix Linux startup flakiness in CI/CD by improving process management
  • Update documentation to reflect current CI/CD state

Changes Made

1. Disabled E2E Tests (Temporary)

  • Commented out E2E test execution in PR build workflow
  • Tests were causing intermittent failures due to timing issues
  • Infrastructure for running tests remains in place for easy re-enabling

2. Fixed Linux Service Startup Issues

  • Replaced nohup with PowerShell's Start-Process for better process management
  • Added immediate process verification after starting services
  • Replaced Windows-specific Test-NetConnection with cross-platform TCP client
  • Added continuous process monitoring during health checks
  • Separated stdout and stderr into different log files for better debugging
  • Enhanced failure diagnostics with comprehensive log output

3. Documentation Updates

  • Updated README to reflect current CI/CD state
  • Added notes about temporarily disabled E2E tests
  • Documented workflow improvements and cross-platform compatibility

Test Plan

  • Verify PR build workflow runs successfully on both Windows and Linux
  • Confirm services start reliably without flaky failures
  • Check that error logs are properly captured on failures
  • Monitor workflow stability over multiple PR builds

JimboSchneider and others added 15 commits June 7, 2025 17:24
- Fixed timing issues in E2E tests by consolidating wait logic
- Improved selector specificity to avoid ambiguous element matches
- Updated README with CI/CD workflow documentation
- Enhanced E2E test instructions for local development

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Set initial version to 0.0.1 reflecting early development state
- Display version in footer on all pages
- Add comprehensive About page with version and system information
- Update navigation to include About page link

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Increase CI wait time to 60s for better service startup reliability
- Update login test assertion to match actual error message format
- Use C# 12 collection expressions throughout codebase
- Add default values to Post model properties for null safety

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
…orkflow

- Switch from bash to PowerShell for cross-platform compatibility
- Update cache action from v3 to v4
- Add proper background process handling for Windows and Linux
- Implement health checks with 120-second timeout for services
- Add process cleanup that runs even on failure
- Capture and upload service logs as artifacts on failure
- Use platform-appropriate path handling

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
- Increase timeout from 120 to 300 seconds for slower CI environments
- Use more descriptive variable names ($siloPid, $blazorPid) for clarity

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Comment out E2E test execution step until tests are ready for reliable execution.
This prevents PR builds from failing due to incomplete E2E test implementation.
- Replace nohup with PowerShell Start-Process for better process management
- Add immediate process verification after starting services
- Replace Windows-specific Test-NetConnection with cross-platform TCP client
- Add continuous process monitoring during health checks
- Separate stdout and stderr into different log files for better debugging
- Enhanced failure diagnostics with comprehensive log output
- Properly handle HTTPS certificate validation differences between platforms
- Consistent PID tracking across Windows and Linux

These changes should significantly reduce flakiness by catching startup
failures immediately and providing better visibility into service failures.
- Clarify that E2E test automation is temporarily disabled
- Document enhanced service management improvements in workflow
- Add cross-platform compatibility note for PowerShell Core usage
- Add E2E test status to known issues section
@Copilot Copilot AI review requested due to automatic review settings June 15, 2025 16:17
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR enhances CI/CD reliability by stabilizing service startup and documenting workflow updates, temporarily disables flaky E2E tests, and adds version information to the UI.

  • Temporarily disabled intermittent E2E tests in the PR build workflow
  • Replaced nohup with PowerShell’s Start-Process, added cross-platform health checks, and improved logging
  • Updated documentation and UI with an About page displaying application and assembly version

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
README.md Clarified E2E test steps, added CI/CD workflow documentation
OrleansBlog/OrleansBlog.csproj Set project version and file/assembly versions
OrleansBlog/Models/Post.cs Initialized non-nullable properties with defaults and required
OrleansBlog/Components/Pages/Post.razor Switched to C# 12 array literal in ParseTags
OrleansBlog/Components/Pages/About.razor New About page showing version and runtime info
OrleansBlog/Components/Layout/NavMenu.razor Added navigation link for the About page
OrleansBlog/Components/Layout/MainLayout.razor & .css Added footer with version, updated link target, and CSS padding
OrleansBlog.Tests/PostServiceTests.cs Updated tests for new array literal syntax in tags
.github/workflows/pr-build.yml Upgraded cache action, improved service startup/health checks, disabled E2E run
Comments suppressed due to low confidence (3)

OrleansBlog/Components/Layout/NavMenu.razor:41

  • The Bootstrap icon class "bi-info-circle-nav-menu" does not match any standard icon; it should likely be "bi-info-circle" for the info-circle icon.
<span class="bi bi-info-circle-nav-menu" aria-hidden="true"></span> About

OrleansBlog.Tests/PostServiceTests.cs:77

  • Tests for CreatePostAsync and UpdatePostAsync do not verify that the Tags property is correctly passed through; consider adding assertions that validate the tags in the returned or updated post.
Tags = ["new", "test"]

OrleansBlog/Components/Pages/About.razor:1

  • The new About page does not have any coverage in existing tests; consider adding a UI or integration test to verify that version and runtime information renders as expected.
@page "/about"

- Add paths-ignore to PR build workflow to skip builds for:
  - Markdown files (*.md)
  - License files
  - .gitignore
  - Documentation directories
  - GitHub issue templates
- Update README to document smart build triggering behavior
- Add CONTRIBUTING.md with guidelines for contributors including PR process

This optimization saves CI/CD resources when only documentation is updated.
@JimboSchneider JimboSchneider merged commit 137ea54 into main Jun 15, 2025
2 checks passed
@JimboSchneider JimboSchneider deleted the fix/e2e-test-reliability branch June 15, 2025 16:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant