feat: add go flaky tests github action #1013
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add core functionality for analyzing test failures from Loki logs: - Query Loki using LogQL for test failure data - Parse and aggregate test failures across branches - Classify tests as flaky based on failure patterns - Generate detailed analysis reports with failure counts and workflow URLs - Configurable time ranges and result limits This initial implementation includes stub interfaces for Git and GitHub functionality that will be added in subsequent PRs.
Remove stub implementations and complexity that will be added in later PRs: - Remove GitClient and GitHubClient interfaces and stubs - Remove FilePath and RecentCommits from FlakyTest struct - Remove repository-directory and skip-posting-issues inputs - Remove author tracking and issue management code paths - Focus purely on Loki querying, log parsing, and flaky test detection This creates a clean foundation for PR2 (Git authors) and PR3 (GitHub issues) to build upon without unnecessary complexity in the initial implementation.
- Add MockLokiClient and MockFileSystem for testing - Test core AnalyzeFailures() functionality with valid Loki response - Test error handling when Loki client fails - Test ActionReport() method for both empty and populated reports - Test utility functions like generateSummary() and FlakyTest.String() - Use proper Loki response format with stream metadata for test data
- Added README.md with detailed usage instructions and how-it-works - Added CHANGELOG.md documenting features and implementation details - Added run-local.sh script for local development and testing - Documentation now matches original PR functionality
- Remove GitHub issue creation and management features - Remove dry run mode references - Remove GitHub CLI and issue template content - Documentation now covers Loki analysis and Git author tracking
- Remove Git history analysis and author tracking features - Remove repository-directory input reference - Remove run-local.sh script usage, use go run directly - Documentation now covers only basic Loki analysis functionality
- Update aggregate.go to consider both 'main' and 'master' branches as indicators of flaky tests - Update README documentation to reflect main/master branch logic - Add comprehensive tests for master branch detection - Remove 'progressive PR structure' and 'technical details' from changelog
- Update action name and description - Rename directories and update all file references - Update module name and build paths - Update documentation and examples
The repository-directory parameter is not needed in the core action and was causing documentation inconsistency.
The github-token parameter is not used in the current implementation and was causing documentation inconsistency.
- Fix prettier formatting for action.yaml, README.md, CHANGELOG.md - Add missing newlines at end of files - Remove test output file that shouldn't be committed
|
A linter is failing and I don't quite understand why. Can a maintainer give me a hand? |
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add core functionality for analyzing test failures from Loki logs.
This PR introduces the foundation of the flaky test detection bot. It is broken up from #998. This is the first PR of 3. The next PRs are dimitarvdimitrov#1 and dimitarvdimitrov#2. They add Git author tracking and GitHub issue management.
Key Features
Why in
grafana/shared-workflowsThis action is designed to work with any Go repository that meets these minimal requirements:
This covers virtually all Grafana Labs repositories today. My plan is to deploy this immediately for grafana/mimir and grafana/backend-enterprise.
Future Extensibility
For non-Grafana repos or different Loki setups, the LogQL query can be made configurable via an additional input parameter, enabling adoption across any organization using Loki for CI/CD observability. Let me know if you think we should go down this route.
The action requires no repository-specific configuration - just provide Loki credentials and the repository name and have the repository closed locally. It automatically discovers test files, identifies recent contributors, and manages GitHub issues.
Testing
Includes a test suite with real Loki response data and golden file testing for reliable parsing of complex log structures.