[8.19] [Security Solution] Get rid of esArchiver in import custom rules integration tests (#229492) #234561
Conversation
There was a problem hiding this comment.
Just a question, to make sure: in my VSCode, checked out for 8.19, in this file, in line 28, I can see already the usage of withSpaceUrl. It looks like this:
.post(withSpaceUrl(DETECTION_ENGINE_RULES_BULK_ACTION, spaceId))
But here it is not there. We spoke offline about different order of making these changes in backports, but if this usage of withSpaceUrl was backported already, why isn't it showing in the file (it should NOT be part of the diff, but it should be visible, am I right?)
There was a problem hiding this comment.
We temporary have utilities/services copies due to SKA. The file you referenced x-pack/solutions/security/test/common/utils/detections_response/rules/delete_all_rules.ts isn't used in the test x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/rule_import_export/trial_license_complete_tier/import_rules.ts. The test uses deleteAllRules from x-pack/solutions/security/test/security_solution_api_integration/config/services/detections_response/rules/delete_all_rules.ts.
There was a problem hiding this comment.
#234642 should fix the duplicated utilities and services issue.
jkelas
left a comment
jkelas
left a comment
There was a problem hiding this comment.
I checked that the backport has been performed correctly. Approving.
…gration tests (elastic#229492) **Resolves: elastic#227454 **Resolves: elastic#224511 **Resolves: elastic#220971 **Resolves: elastic#220536 **Resolves: elastic#217903 **Resolves: elastic#205777 This PR fixes import custom rules flaky integration tests. This is done by refactoring the tests and getting rid of `esArchiver`. Investigation of the flakes specified in the tickets above revealed that the culprit is `esArchiver`. It removes the target indexes before loading the documents and applies migrations. In fact it turns out it removes just installed mock prebuilt rules package forcing installing a real prebuilt rules package from the production EPR. Where usage of production EPR has been known to cause flakiness. On top of just getting rid of `esArchiver` this PR refactors the importing custom rules integrations tests. The main goal is to make the tests structure clear and transparent. Previously flaky tests were moved to the different sections. And some of the tests testing Kibana v7.14 connectors were removed since `esArchiver` performs connectors migration so importing rules tests interact with the migrated connections. - ✅ [ESS 200 runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8883) - ✅ [Serverless 200 runs](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8884) - Add `--xpack.fleet.registryUrl=http://non-existent` to [config.base.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_api_integration/config/ess/config.base.ts) for ESS or [config.base.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_api_integration/config/serverless/config.base.ts) for Serverless. - run import custom rules integration tests in the `main` branch and observe test failures - run import custom rules integration tests in this PR and observe all tests have passed (cherry picked from commit 6403fd4)
maximpn
force-pushed
the
backport/8.19/pr-229492
branch
from
d482e90 to
f298c3a
Compare
💚 Build Succeeded
Metrics [docs]
History
|
3 participants
Backport
This will backport the following commits from
mainto8.19:Questions ?
Please refer to the Backport tool documentation