Gap auto fill scheduler UI and API #244719

nkhristinin · 2025-12-01T10:05:21Z

Gap auto fill scheduler UI and API

Overview

This PR introduces the UI and APIs for the Gap auto fill scheduler.

It completes the full CRUD API (Read, Update, Delete) for the gap fill scheduler configuration.

A new Rules Settings modal and Gap scheduler logs flyout are introduced in the rule page to provide users with a purpose to enable/disable scheduler and see the logs.

What is Gap auto fill scheduler?

Rule can have gaps - period of time when rule didn't execute.

To resolve a gap, we can initiate a manual rule run (backfill) for the specific rule and time range. It will execute rule and then update gaps (make them "filled")

The Gap auto fill scheduler automates this. It runs on a defined schedule, finds rules that have gaps, and automatically schedules the backfill runs to fill them.

The feature is gated by a Enterprise+ license.

New APIs

Get Gap Auto Fill Scheduler

Endpoint:
GET /internal/alerting/rules/gaps/auto_fill_scheduler/{id}

Update Gap Auto Fill Scheduler

Endpoint:
PUT /internal/alerting/rules/gaps/auto_fill_scheduler/{id}

Delete Gap Auto Fill Scheduler

Endpoint:
DELETE /internal/alerting/rules/gaps/auto_fill_scheduler/{id}

Get Gap Auto Fill Scheduler Execution Logs

Endpoint:
POST /internal/alerting/rules/gaps/auto_fill_scheduler/{id}/logs

Cleanup

Update (when disable) and deletion API handles cleanup of the associated task manager task and delete any backfills created by this scheduler.

Event Log Statuses

We introduce a new status in the scheduler execution log flow.

NO_GAPS: Added to the status field. This status is used when the scheduler successfully runs but finds no eligible rules with gaps to process.

UI Changes

New Rules Settings Modal

A "Rules Settings" button is added to the Rules page.
This modal allows users to manage the Gap auto fill scheduler configuration (enable/disable now).

Gap scheduler logs flyout

A "logs" link in the settings modal opens a flyout displaying the scheduler's execution logs from the Event Log.
The flyout provides details on run status, and log message.

How to Test

Feature Flag

The UI and API endpoints are controlled by the following feature flag, which must be enabled in kibana.dev.yml:

xpack.alerting.gapAutoFillScheduler.enabled: true
xpack.securitySolution.enableExperimental: [ 'gapAutoFillSchedulerEnabled']

Ensure you have rules with gaps

There are two ways to create gaps:

Manual method:
Create and enable a security rule with a 1-minute interval and 0-second lookback.
After the first run, disable the rule, wait 5 minutes, and then enable it again you should execution error about gaps, and see the gap in the gaps table in the execution tab.
Using the this tool:
Run the following command to generate multiple rules and gaps (100 rules, 10 gaps each, 30m interval rule, and remove all rules before):
```
npm run start -- rules --rules 100 -c -g 10 -i "30m"
```

Go to rules page
Open rule settings modal window and enable gap auto fill scheduler.
On rule monitoring table observe that are some rules are in progress for gap filling.
From the rule settings modal window open the logs flyout and observe the logs

…te --fix'

…no-cache --fix'

… backfill-iniator

… src/core/server/integration_tests/ci_checks'

… backfill-iniator

…no-cache --fix'

…to gap-auto-fill-task

…te --fix'

…to gap-auto-fill-task

…no-cache --fix'

…to gap-auto-fill-task

nkhristinin · 2025-12-02T15:04:43Z

@elasticmachine merge upstream

nkhristinin · 2025-12-03T07:10:24Z

@elasticmachine merge upstream

…bana into gap-auto-fill-task-api-ui

nkhristinin · 2025-12-03T11:03:03Z

@elasticmachine merge upstream

denar50 · 2025-12-03T11:42:39Z

.../security_solution/public/detection_engine/rule_gaps/components/gap_auto_fill_logs/index.tsx

+
+  return (
+    <>
+      {isOpen && (


As it is now, this component is always rendering when the settings modal opens, even if the user never opens the flyout by clicking the logs link. As a result, it is making 4 unnecessary post requests to fetch logs.
A solution for this could be to move away the hooks that fetch the logs to their own component and have an outer component GapAutoFillLogsFlyout which decides whether to render the component that calls the hooks when open, or render nothing when closed.

denar50 · 2025-12-03T12:05:53Z

I have found a strange behavior on a fresh installation. If I open the settings and click on save, without activating the scheduler, it activates automatically (see the video below). My expectation as a user would be that if I do not change any settings, they remain the same when I save/cancel/close the modal.

Screen.Recording.2025-12-03.at.13.03.12.mov

denar50 · 2025-12-03T12:54:10Z

...security_solution/public/detection_engine/rule_gaps/components/rule_settings_modal/index.tsx

+            <EuiSpacer size="m" />
+
+            <EuiFormRow>
+              <EuiSwitch


Users can interact with this toggle if the request to get the current gap auto fill configuration takes a long time. Consider disabling the toggle and the save button while that request loads.

denar50 · 2025-12-03T13:02:24Z

.../server/application/gap_auto_fill_scheduler/methods/update/update_gap_auto_fill_scheduler.ts

+  const uniqueRuleTypeIds = new Set(updatedRuleTypes.map(({ type }) => type));
+
+  for (const ruleTypeId of uniqueRuleTypeIds) {
+    context.ruleTypeRegistry.get(ruleTypeId);


You are not doing anything with the result of this line. Is this meant to validate that the rule types are valid?

denar50 · 2025-12-03T13:25:01Z

.../server/application/gap_auto_fill_scheduler/methods/update/update_gap_auto_fill_scheduler.ts

+    map.set(toRuleTypeKey(ruleType), ruleType);
+  }
+  if (incoming) {
+    for (const ruleType of incoming) {


Instead of having two for blocks doing the same thing, consider merging both arrays at the beginning.

…bana into gap-auto-fill-task-api-ui

nkhristinin · 2025-12-03T15:39:06Z

@denar50 thanks for those comments, UI suggestion very good. I addressed them all

denar50 · 2025-12-03T13:38:01Z

...security_solution/public/detection_engine/rule_gaps/components/rule_settings_modal/index.tsx

+      if (!gapAutoFillScheduler) {
+        await createMutation.mutateAsync();
+      } else {
+        await updateMutation.mutateAsync({ ...gapAutoFillScheduler, enabled });


You're always updating even when the user doesn't change anything. I took a look at the request itself and it is a bit expensive, currently taking almost 2 seconds when the rule is enabled (with the call to taskManager.ensureScheduled taking most of the time). Consider either preventing that call here or, even better, short circuiting in the backend to avoid doing unnecessary work.

denar50 · 2025-12-03T15:54:14Z

.../server/application/gap_auto_fill_scheduler/methods/delete/delete_gap_auto_fill_scheduler.ts

+
+    await taskManager.removeIfExists(scheduledTaskId);
+
+    await soClient.delete(GAP_AUTO_FILL_SCHEDULER_SAVED_OBJECT_TYPE, params.id);


It seems that removing the saved object is the point of no return here. It might be worth to do it after deleting the backfills as the last step.

denar50 · 2025-12-03T16:00:22Z

...g/server/application/gap_auto_fill_scheduler/methods/get/get_gap_auto_fill_scheduler.test.ts

+
+  beforeEach(() => {
+    jest.resetAllMocks();
+    rulesClient = new RulesClient({


This bunch of code repeats across the test files of the CRUD logic. Consider moving this initialization logic to one place within the ... gap_auto_fill_scheduler/methods folder. You can have a function that builds the rules client along with all the other clients you initialize in lines 30-37.

denar50 · 2025-12-03T16:03:44Z

...erting/server/application/gap_auto_fill_scheduler/methods/get/get_gap_auto_fill_scheduler.ts

+
+    // Authorization check - we need to check if user has permission to get
+    // For gap fill auto scheduler, we check against the rule types it manages
+    const ruleTypes = result.attributes.ruleTypes || [];


Why would this be falsy?

denar50 · 2025-12-03T16:06:22Z

...ver/application/gap_auto_fill_scheduler/methods/get_logs/get_gap_auto_fill_scheduler_logs.ts

+
+    // Authorization check - we need to check if user has permission to get logs
+    // For gap fill auto scheduler, we check against the rule types it manages
+    const ruleTypes = schedulerSO.attributes.ruleTypes || [];


same here, why would this be falsy?

denar50 · 2025-12-03T16:10:04Z

...plugins/shared/alerting/server/application/gap_auto_fill_scheduler/methods/get_logs/utils.ts

+import type { IValidatedEventInternalDocInfo } from '@kbn/event-log-plugin/server';
+import type { GapAutoFillSchedulerLogEntry } from './types';
+
+export const formatGapAutoFillSchedulerLogEntry = (


Might be worth adding a unit test for this utils function.

denar50 · 2025-12-03T16:16:18Z

...er/application/gap_auto_fill_scheduler/methods/update/update_gap_auto_fill_scheduler.test.ts

I think it would be good to add a test where you have an autofill schedule with 3 rule types, and the update request has 1 of those rule types. The functionality still makes sure that all the rule types in the original SO, and not just the one in the parameters, gets updated by making a union of the types.

elasticmachine · 2025-12-03T17:25:46Z

💔 Build Failed

Buildkite Build
Commit: 8b7970f

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #80 / Maps endpoints apis search ES|QL should return getValues response in expected shape
[job] [logs] FTR Configs #80 / Maps endpoints apis search ES|QL should return getValues response in expected shape

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	8448	8454	+6

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`alerting`	827	829	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	11.1MB	11.1MB	+14.8KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`alerting`	64	65	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`alerting`	23.9KB	24.0KB	+102.0B
`securitySolution`	166.8KB	166.8KB	+31.0B
total			+133.0B

Unknown metric groups

API count

id	before	after	diff
`alerting`	864	866	+2

History

cc @nkhristinin

ymao1 · 2025-12-03T18:19:36Z

x-pack/platform/plugins/shared/alerting/server/lib/license_state.ts

    }
  }

+  public ensureLicenseForGapAutoFillScheduler() {


update unit tests

ymao1 · 2025-12-03T18:22:15Z

...ing/server/routes/gaps/apis/gap_auto_fill_schedule/get/get_auto_fill_scheduler_route.test.ts

+    });
+  });
+
+  test('ensures the license allows for getting gap fill auto scheduler', async () => {


should update tests to include the custom license check you added

ymao1 · 2025-12-03T18:27:47Z

...atform/plugins/shared/alerting/common/routes/gaps/apis/gap_auto_fill_scheduler/schemas/v1.ts

+    name: schema.string(),
+    enabled: schema.boolean(),
+    gap_fill_range: schema.string(),
+    max_backfills: schema.number({ min: 1, max: 5000 }),


should this be

Suggested change

max_backfills: schema.number({ min: 1, max: 5000 }),

max_backfills: schema.number(maxBackfills),

ymao1 · 2025-12-03T18:28:30Z

...atform/plugins/shared/alerting/common/routes/gaps/apis/gap_auto_fill_scheduler/schemas/v1.ts

+    enabled: schema.boolean(),
+    gap_fill_range: schema.string(),
+    max_backfills: schema.number({ min: 1, max: 5000 }),
+    num_retries: schema.number({ min: 1 }),


should this be

Suggested change

num_retries: schema.number({ min: 1 }),

num_retries: schema.number(numRetries),

ymao1 · 2025-12-03T18:31:09Z

...rver/routes/gaps/apis/gap_auto_fill_schedule/update/update_auto_fill_scheduler_route.test.ts

+    expect(licenseState.ensureLicenseForGapAutoFillScheduler).toHaveBeenCalled();
+  });
+
+  test('respects license failures', async () => {


should update test for ensureLicenseForGapAutoFillScheduler

ymao1 · 2025-12-03T18:56:23Z

...ver/application/gap_auto_fill_scheduler/methods/get_logs/get_gap_auto_fill_scheduler_logs.ts

+
+  try {
+    // Get the scheduler saved object to access ruleTypes for authorization
+    const schedulerSO = await context.unsecuredSavedObjectsClient.get<GapAutoFillSchedulerSO>(


This GET logic is repeated in multiple places. It would be useful to create a library function that encapsulates getting the SO, checking for errors and performing the authorization check and reusing that in all of these functions.

ymao1 · 2025-12-03T19:05:40Z

.../server/application/gap_auto_fill_scheduler/methods/update/update_gap_auto_fill_scheduler.ts

+    await taskManager.removeIfExists(updatedSo.id);
+
+    if (updatedEnabled) {
+      await taskManager.ensureScheduled(


We will be working on a bug fix to update TM tasks with API keys: #244918. When that is done, you should just be able to update the schedule of the existing task and not have to reschedule it.

ymao1 · 2025-12-03T19:06:42Z

.../server/application/gap_auto_fill_scheduler/methods/update/update_gap_auto_fill_scheduler.ts

+          id: updatedSo.id,
+          taskType: GAP_AUTO_FILL_SCHEDULER_TASK_TYPE,
+          schedule: updatedSchedule,
+          scope: updatedScope ?? [],


Does the task scope really matter? The scope is stored inside the saved object already. If we don't have to update the task scope, you should be able to just use the existing updateSchedule functions that task manager provides (once the above mentioned bug is fixed).

ymao1 · 2025-12-03T19:13:52Z

x-pack/platform/plugins/shared/alerting/server/backfill_client/backfill_client.ts

+    internalSavedObjectsRepository,
+    eventLogClient,
+    eventLogger,
+    actionsClient,


These new fields are not used for deleteBackfillForRules so should not be part of the method definition because it makes it confusing. They're all optional fields anyway, so you can exclude them when calling deleteAdHocRunsAndTasks from this function.

ymao1 · 2025-12-03T19:16:29Z

x-pack/platform/plugins/shared/alerting/server/backfill_client/backfill_client.ts

+    if (adHocRuns.length === 0) return;
+
+    // Prepare backfill metadata for gap updates before deleting SOs
+    const backfillsForGapUpdate =


suggest creating a const like const canUpdateGaps = shouldUpdateGaps && actionsClient && internalSavedObjectsRepository && eventLogClient and reusing.

nkhristinin and others added 30 commits October 6, 2025 15:24

Add initator field to backfill

31c0e58

[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…

b154793

…te --fix'

[CI] Auto-commit changed files from 'node scripts/eslint_all_files --…

26daa71

…no-cache --fix'

Fixes types

4c14e11

Merge branch 'backfill-iniator' of github.com:nkhristinin/kibana into…

8e2c648

… backfill-iniator

[CI] Auto-commit changed files from 'node scripts/jest_integration -u…

65ed6f5

… src/core/server/integration_tests/ci_checks'

update types

213c9fb

Merge branch 'backfill-iniator' of github.com:nkhristinin/kibana into…

977f06e

… backfill-iniator

Fix more types

c3e43e7

fix more types

4e19aa2

Merge branch 'main' into backfill-iniator

981580e

Fix tests

fb677d6

Merge branch 'backfill-iniator' of github.com:nkhristinin/kibana into…

a2853d8

… backfill-iniator

Add task, api, event log mappings

a74fce3

[CI] Auto-commit changed files from 'node scripts/eslint_all_files --…

b65b5aa

…no-cache --fix'

Fix how task get space from request

25c7ba1

Merge branch 'gap-auto-fill-task' of github.com:nkhristinin/kibana in…

bd19d50

…to gap-auto-fill-task

Return default value for space

14839b1

use backfill initator constant

de66f16

Some fixes

163655b

fix some unit tests

d84bd51

Merge branch 'main' into gap-auto-fill-task

e6e0d53

[CI] Auto-commit changed files from 'node scripts/check_mappings_upda…

5f2be71

…te --fix'

fixes

62332f2

Merge branch 'main' into gap-auto-fill-task

0f9f792

Merge branch 'gap-auto-fill-task' of github.com:nkhristinin/kibana in…

4fc5880

…to gap-auto-fill-task

[CI] Auto-commit changed files from 'node scripts/eslint_all_files --…

6e1efdf

…no-cache --fix'

Fix tests and types

63a4f54

Change tests and fix how we check overlapps

7f641e4

Merge branch 'gap-auto-fill-task' of github.com:nkhristinin/kibana in…

bd4c92d

…to gap-auto-fill-task

nkhristinin added 3 commits December 2, 2025 16:00

fix tests

71a7b7f

fix tests

3fba6fd

fix cypress tests

2784fc9

Merge branch 'main' into gap-auto-fill-task-api-ui

fe01d5f

denar50 self-requested a review December 2, 2025 16:45

elasticmachine and others added 4 commits December 3, 2025 08:10

Merge branch 'main' into gap-auto-fill-task-api-ui

0d7f1a8

change licencse to enterprise

afb28f1

remove serverless tag for basic license cypress tests

456d93c

Merge branch 'gap-auto-fill-task-api-ui' of github.com:nkhristinin/ki…

1e61142

…bana into gap-auto-fill-task-api-ui

Merge branch 'main' into gap-auto-fill-task-api-ui

a9272db

denar50 reviewed Dec 3, 2025

View reviewed changes

nkhristinin added 7 commits December 3, 2025 16:13

reduce amount of requests

bb04e68

api threat enabled param on creation

4168a0b

Disable button on save

0ed43aa

also disabeld when load gap scheduler config

ea7c2e6

simplify buildRuleTypeUnion

95ee9e1

remove serverlss tag for cy test

c38a154

Merge branch 'gap-auto-fill-task-api-ui' of github.com:nkhristinin/ki…

8b7970f

…bana into gap-auto-fill-task-api-ui

nkhristinin requested a review from denar50 December 3, 2025 15:39

denar50 reviewed Dec 3, 2025

View reviewed changes

ymao1 reviewed Dec 3, 2025

View reviewed changes


		await taskManager.removeIfExists(scheduledTaskId);

		await soClient.delete(GAP_AUTO_FILL_SCHEDULER_SAVED_OBJECT_TYPE, params.id);

	max_backfills: schema.number({ min: 1, max: 5000 }),
	max_backfills: schema.number(maxBackfills),

	num_retries: schema.number({ min: 1 }),
	num_retries: schema.number(numRetries),

Gap auto fill scheduler UI and API #244719

Are you sure you want to change the base?

Gap auto fill scheduler UI and API #244719

Conversation

nkhristinin commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Gap auto fill scheduler UI and API

Overview

What is Gap auto fill scheduler?

New APIs

Cleanup

Update (when disable) and deletion API handles cleanup of the associated task manager task and delete any backfills created by this scheduler.

Event Log Statuses

UI Changes

How to Test

Uh oh!

nkhristinin commented Dec 2, 2025

Uh oh!

nkhristinin commented Dec 3, 2025

Uh oh!

nkhristinin commented Dec 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

denar50 commented Dec 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nkhristinin commented Dec 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

History

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nkhristinin commented Dec 1, 2025 •

edited

Loading

elasticmachine commented Dec 3, 2025 •

edited

Loading