[MINOR] feat(CI): Introducing surefire.rerunFailingTestsCount to identify Flakes tests.

[zhengchenyu]

What changes were proposed in this pull request?

We can find many flaky test, it is not conducive to review code.

We can introduce surefire.rerunFailingTestsCount=2, the test is considered failed when it fails three times, or Flaskes.

[zhengchenyu]

@jerqi @zuston

Do you think we need to introduce rerunFailingTestsCount?

[github-actions]

Test Results

 2 657 files  +10   2 657 suites  +10   5h 30m 24s ⏱️ + 3m 50s
   946 tests ± 0     945 ✅ + 1   1 💤 ±0  0 ❌ ±0 
11 799 runs  +10  11 784 ✅ +11  15 💤 ±0  0 ❌ ±0 

Results for commit 4c5a174. ± Comparison against base commit 7049569.

[rickyma]

This will cause issues to be hidden. We need to find out the flaky tests and fix them. This is what UTs should do.

[zhengchenyu]

@rickyma

In fact, flaky test will not be hidden, they will show in logs. Here are hadoop test log:

[WARNING] Flakes: 
[WARNING] org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks.testSetRepIncWithUnderReplicatedBlocks(org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks)
[ERROR]   Run 1: TestUnderReplicatedBlocks.testSetRepIncWithUnderReplicatedBlocks:80 » TestTimedOut
[INFO]   Run 2: PASS

For now, when the some test failed, we will manually confirm whether it is flaky. If it is flaky, we still merged to master.
I'm afraid that some serious errors may be mistaken for flaky due to human negligence.

[rickyma]

If all the tests finally pass, who will still check the logs and look for the flaky tests? I'm not sure.