Skip to content

Commit 374b6a5

Browse files
authored
[Security Solution] Parallelize slow Rule Management tests (elastic#234930)
**Resolves: elastic#232644 This PR splits long-running API integration and Jest configs into smaller configs to reduce overall runtime. We can’t rely on Buildkite’s `parallelism` parameter here, since it’s only works for Cypress steps. ## Changes - **API Integration tests** - Split the prebuilt rules config into 3 smaller configs. - **Rule Upgrade Field Tests (Jest)** - Merged `common_fields` and `type_specific_fields` dirs into a single `fields` directory. - Replaced 2 configs with 4 smaller ones. - Implemented round-robin logic to evenly distribute tests across the 4 configs. - **Alternative**: manually distribute tests into numbered directories. - *Pros*: simpler setup. - *Cons*: requires deciding where each new test should live when adding new fields. - Reviewers, tell me if adding this distribution logic is an overkill in your opinion.
1 parent d4dc5f5 commit 374b6a5

File tree

51 files changed

+335
-187
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+335
-187
lines changed

.buildkite/ftr_security_serverless_configs.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,9 @@ enabled:
8787
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/rule_update/basic_license_essentials_tier/configs/serverless.config.ts
8888
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/serverless_essentials_tier.config.ts
8989
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_disabled/configs/serverless_essentials_tier.config.ts
90-
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/configs/serverless.config.ts
90+
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/customization/configs/serverless.config.ts
91+
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/configs/serverless.config.ts
92+
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_notifications/configs/serverless.config.ts
9193
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/diffable_rule_fields/common_fields/configs/serverless.config.ts
9294
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/diffable_rule_fields/type_specific_fields/configs/serverless.config.ts
9395
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/ml_disabled/configs/serverless_essentials_tier.config.ts

.buildkite/ftr_security_stateful_configs.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,9 @@ enabled:
7373
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/edge_cases/ess_air_gapped_with_bundled_large_package.config.ts
7474
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_disabled/configs/ess_basic_license.config.ts
7575
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/common/configs/edge_cases/ess_trial_license.config.ts
76-
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/configs/ess.config.ts
76+
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/customization/configs/ess.config.ts
77+
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/configs/ess.config.ts
78+
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_notifications/configs/ess.config.ts
7779
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/diffable_rule_fields/common_fields/configs/ess.config.ts
7880
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/customization_enabled/upgrade_prebuilt_rules/diffable_rule_fields/type_specific_fields/configs/ess.config.ts
7981
- x-pack/solutions/security/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/ml_disabled/configs/ess_basic_license.config.ts

x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/pages/rule_management/__integration_tests__/rules_upgrade/upgrade_rule_after_preview/common_fields/jest.integration.config.js

Lines changed: 0 additions & 17 deletions
This file was deleted.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
/*
2+
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
* or more contributor license agreements. Licensed under the Elastic License
4+
* 2.0; you may not use this file except in compliance with the Elastic License
5+
* 2.0.
6+
*/
7+
8+
/* eslint-disable import/no-nodejs-modules */
9+
import fs from 'fs';
10+
11+
// Discover all test files
12+
const allTestFiles = fs
13+
.readdirSync(__dirname, { recursive: true })
14+
.filter((f) => f.endsWith('.test.ts'));
15+
16+
/**
17+
* Distributes all field test files evenly using round-robin across available configs.
18+
*/
19+
function getTestsForConfig({ testsDirectory, groupNumber, totalGroups }) {
20+
const groupIndex = groupNumber - 1;
21+
22+
const testFiles = allTestFiles.filter((file) => file.includes(testsDirectory));
23+
const testFilesForConfig = testFiles.filter((file, index) => index % totalGroups === groupIndex);
24+
25+
console.log(
26+
`Rule Upgrade - Fields integration tests. ${testsDirectory}, config_${groupNumber}: Running ${testFilesForConfig.length} test files:`,
27+
testFilesForConfig
28+
);
29+
30+
return testFilesForConfig;
31+
}
32+
33+
/**
34+
* Base Jest configuration shared by all field test configs
35+
*/
36+
export function createFieldTestingConfig({ testsDirectory, groupNumber, totalGroups }) {
37+
const testFiles = getTestsForConfig({
38+
testsDirectory,
39+
groupNumber,
40+
totalGroups,
41+
});
42+
43+
return {
44+
preset: '@kbn/test/jest_integration',
45+
rootDir: '../../../../../../../../../../../../../../../..',
46+
roots: [
47+
'<rootDir>/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/pages/rule_management',
48+
],
49+
testMatch: testFiles.map((file) => `**/${file}`),
50+
openHandlesTimeout: 0,
51+
forceExit: true,
52+
};
53+
}
Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@
55
* 2.0.
66
*/
77

8-
import { mockAvailableDataViews } from '../../test_utils/rule_upgrade_flyout';
9-
import { assertRuleUpgradePreview } from '../../test_utils/assert_rule_upgrade_preview';
10-
import { assertRuleUpgradeAfterReview } from '../../test_utils/assert_rule_upgrade_after_review';
11-
import { assertFieldValidation } from '../../test_utils/assert_field_validation';
12-
import { assertDiffAfterSavingUnchangedValue } from '../../test_utils/assert_diff_after_saving_unchanged_value';
8+
import { mockAvailableDataViews } from '../../../test_utils/rule_upgrade_flyout';
9+
import { assertRuleUpgradePreview } from '../../../test_utils/assert_rule_upgrade_preview';
10+
import { assertRuleUpgradeAfterReview } from '../../../test_utils/assert_rule_upgrade_after_review';
11+
import { assertFieldValidation } from '../../../test_utils/assert_field_validation';
12+
import { assertDiffAfterSavingUnchangedValue } from '../../../test_utils/assert_diff_after_saving_unchanged_value';
1313

1414
describe('Upgrade diffable rule "alert_suppression" (query rule type) after preview in flyout', () => {
1515
beforeAll(() => {
Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@
55
* 2.0.
66
*/
77

8-
import { assertRuleUpgradePreview } from '../../test_utils/assert_rule_upgrade_preview';
9-
import { assertRuleUpgradeAfterReview } from '../../test_utils/assert_rule_upgrade_after_review';
10-
import { assertDiffAfterSavingUnchangedValue } from '../../test_utils/assert_diff_after_saving_unchanged_value';
8+
import { assertRuleUpgradePreview } from '../../../test_utils/assert_rule_upgrade_preview';
9+
import { assertRuleUpgradeAfterReview } from '../../../test_utils/assert_rule_upgrade_after_review';
10+
import { assertDiffAfterSavingUnchangedValue } from '../../../test_utils/assert_diff_after_saving_unchanged_value';
1111

1212
describe('Upgrade diffable rule "building_block" (query rule type) after preview in flyout', () => {
1313
const ruleType = 'query';
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
/*
2+
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
* or more contributor license agreements. Licensed under the Elastic License
4+
* 2.0; you may not use this file except in compliance with the Elastic License
5+
* 2.0.
6+
*/
7+
8+
import { createFieldTestingConfig } from '../../base.jest.integration.config';
9+
10+
export default createFieldTestingConfig({
11+
testsDirectory: 'common',
12+
groupNumber: 1,
13+
totalGroups: 2,
14+
});
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
/*
2+
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
* or more contributor license agreements. Licensed under the Elastic License
4+
* 2.0; you may not use this file except in compliance with the Elastic License
5+
* 2.0.
6+
*/
7+
8+
import { createFieldTestingConfig } from '../../base.jest.integration.config';
9+
10+
export default createFieldTestingConfig({
11+
testsDirectory: 'common',
12+
groupNumber: 2,
13+
totalGroups: 2,
14+
});
Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,12 @@
55
* 2.0.
66
*/
77

8-
import { DataSourceType } from '../../../../../../../../../common/api/detection_engine';
9-
import { mockAvailableDataViews } from '../../test_utils/rule_upgrade_flyout';
10-
import { assertRuleUpgradePreview } from '../../test_utils/assert_rule_upgrade_preview';
11-
import { assertRuleUpgradeAfterReview } from '../../test_utils/assert_rule_upgrade_after_review';
12-
import { assertDiffAfterSavingUnchangedValue } from '../../test_utils/assert_diff_after_saving_unchanged_value';
13-
import { assertFieldValidation } from '../../test_utils/assert_field_validation';
8+
import { DataSourceType } from '../../../../../../../../../../common/api/detection_engine';
9+
import { mockAvailableDataViews } from '../../../test_utils/rule_upgrade_flyout';
10+
import { assertRuleUpgradePreview } from '../../../test_utils/assert_rule_upgrade_preview';
11+
import { assertRuleUpgradeAfterReview } from '../../../test_utils/assert_rule_upgrade_after_review';
12+
import { assertDiffAfterSavingUnchangedValue } from '../../../test_utils/assert_diff_after_saving_unchanged_value';
13+
import { assertFieldValidation } from '../../../test_utils/assert_field_validation';
1414

1515
describe('Upgrade diffable rule "data_source" (query rule type) after preview in flyout', () => {
1616
beforeAll(() => {
Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@
55
* 2.0.
66
*/
77

8-
import { assertRuleUpgradePreview } from '../../test_utils/assert_rule_upgrade_preview';
9-
import { assertRuleUpgradeAfterReview } from '../../test_utils/assert_rule_upgrade_after_review';
10-
import { assertDiffAfterSavingUnchangedValue } from '../../test_utils/assert_diff_after_saving_unchanged_value';
11-
import { assertFieldValidation } from '../../test_utils/assert_field_validation';
8+
import { assertRuleUpgradePreview } from '../../../test_utils/assert_rule_upgrade_preview';
9+
import { assertRuleUpgradeAfterReview } from '../../../test_utils/assert_rule_upgrade_after_review';
10+
import { assertDiffAfterSavingUnchangedValue } from '../../../test_utils/assert_diff_after_saving_unchanged_value';
11+
import { assertFieldValidation } from '../../../test_utils/assert_field_validation';
1212

1313
describe('Upgrade diffable rule "description" (query rule type) after preview in flyout', () => {
1414
const ruleType = 'query';

0 commit comments

Comments
 (0)