Skip to content

fix: Mark flaky tests to rerun on failure #610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 3, 2025

Conversation

bhimrazy
Copy link
Collaborator

@bhimrazy bhimrazy commented Jun 3, 2025

What does this PR do ?

Mark specific tests as flaky to allow them to rerun automatically upon failure, improving test reliability.

Partially covers: #437

Functions falling under this:

  • tests/streaming/test_dataset.py::test_subsample_streaming_dataset_with_token_loader
  • tests/streaming/test_dataset.py::test_dataset_for_text_tokens_distributed_num_workers_end_to_end

Copy link

codecov bot commented Jun 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80%. Comparing base (1a0f48c) to head (47333f7).
Report is 1 commits behind head on main.

Additional details and impacted files
@@         Coverage Diff         @@
##           main   #610   +/-   ##
===================================
  Coverage    80%    80%           
===================================
  Files        43     43           
  Lines      6463   6463           
===================================
  Hits       5156   5156           
  Misses     1307   1307           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tchaton tchaton merged commit 01a33b0 into Lightning-AI:main Jun 3, 2025
32 checks passed
@bhimrazy bhimrazy deleted the fix/mark-tests-flaky branch June 4, 2025 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants