[8.19](backport #47154) beatconverter: Ignore disabled elasticsearch output (#47559)

* beatconverter: Ignore disabled elasticsearch output (#47154)

## Proposed commit message
Similar to the logstash output:
https://github.com/elastic/beats/blob/52c5045638c8116550dba017bb455802577a5513/libbeat/otelbeat/beatconverter/beatconverter.go#L82-L85

Fixes #46113

## How to test this PR locally

filebeat.yaml:
```yaml
filebeat.inputs:
  - type: filestream
    id: my-filestream-id
    enabled: true
    paths:
      - /path/to/file.json

output.elasticsearch:
  enabled: false
  hosts: ["localhost:9200"]
```

compile: `cd x-pack/filebeat/ && mage buildOtel`

test with:
1. `x-pack/filebeat/filebeat -c filebeat.yaml -e`
2. `x-pack/filebeat/filebeat otel --config filebeat-task.yaml`

## Related issues
- Closes #46113

## Logs

### Before
```
2025-10-16T11:39:52.274+0200	DEBUG	elasticsearch/config_otel.go:222	Failed to get queue size: missing field accessing 'queue'
2025-10-16T11:39:52.298+0200	info	instance/beat.go:178	Home path: [.] Config path: [.] Data path: [./data] Logs path: [./logs]	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.298+0200	info	instance/beat.go:186	Beat ID: 440437c7-3a3f-409a-ba2e-5aa89d0ab790	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.324+0200	info	publisher	pipeline/module.go:105	Beat name: laptop	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.325+0200	error	modules	fileset/modules.go:157	Not loading modules. Module directory not found: module	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.325+0200	warn	beater/filebeat.go:174	Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.325+0200	info	builders/extension.go:50	Development component. May change in the future.	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "beatsauth", "otelcol.component.kind": "extension"}
2025-10-16T11:39:52.330+0200	info	[email protected]/service.go:222	Starting otel...	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "Version": "9.3.0", "NumCPU": 12}
2025-10-16T11:39:52.330+0200	info	extensions/extensions.go:41	Starting extensions...	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}}
2025-10-16T11:39:52.330+0200	info	extensions/extensions.go:45	Extension is starting...	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "beatsauth", "otelcol.component.kind": "extension"}
2025-10-16T11:39:52.331+0200	info	extensions/extensions.go:62	Extension started.	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "beatsauth", "otelcol.component.kind": "extension"}
2025-10-16T11:39:52.331+0200	info	[email protected]/service.go:245	Everything is ready. Begin running and processing data.	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}}
2025-10-16T11:39:52.331+0200	info	fbreceiver/receiver.go:27	starting filebeat receiver	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.332+0200	info	filebeat	memlog/store.go:134	Finished loading transaction log file for '/home/orestis/src/beats/data/registry/filebeat'. Active transaction id=0	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "store": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.333+0200	warn	beater/filebeat.go:406	Filebeat is unable to load the ingest pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the ingest pipelines or are using Logstash pipelines, you can ignore this warning.	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.333+0200	info	registrar	registrar/registrar.go:103	States Loaded from registrar: 0	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.333+0200	info	crawler	beater/crawler.go:76	Loading Inputs: 1	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.334+0200	info	crawler	beater/crawler.go:148	Starting input (ID: 1771212043283374986)	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.334+0200	info	crawler	beater/crawler.go:111	Loading and starting Inputs completed. Enabled inputs: 1	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.334+0200	info	input.filestream	compat/compat.go:141	Input 'filestream' starting	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "id": "my-filestream-id", "ecs.version": "1.6.0"}
2025-10-16T11:39:52.335+0200	info	input.filestream.metric_registry	inputmon/input.go:182	registering	{"resource": {"service.instance.id": "1f0db8b8-b116-48be-8dfc-bb2aff2578e6", "service.name": "otel", "service.version": "9.3.0"}, "otelcol.component.id": "filebeatreceiver", "otelcol.component.kind": "receiver", "otelcol.signal": "logs", "service.name": "filebeat", "id": "my-filestream-id", "registry_id": "my-filestream-id", "input_id": "my-filestream-id", "input_type": "filestream", "ecs.version": "1.6.0"}
```

### After
```
x-pack/filebeat/filebeat otel --config filebeat-task.yaml
Error: invalid configuration: no exporter configuration specified in config
service::pipelines::logs: must have at least one exporter
Usage:
  filebeat otel [flags]
  filebeat otel [command]

Available Commands:
  inspect     Run this command to inspect the OTel configuration translated from the Beats config

Flags:
      --config string   path to filebeat config file (default "filebeat-otel.yml")
  -h, --help            help for otel

Global Flags:
  -E, --E setting=value              Configuration overwrite
  -M, --M setting=value              Module configuration overwrite
  -c, --c string                     Configuration file, relative to path.config (default "filebeat.yml")
  -d, --d string                     Enable certain debug selectors
  -e, --e                            Log to stderr and disable syslog/file output
      --environment environmentVar   set environment being ran in (default default)
      --path.config string           Configuration path
      --path.data string             Data path
      --path.home string             Home path
      --path.logs string             Logs path
      --plugin pluginList            Load additional plugins
      --strict.perms                 Strict permission checking on config files (default true)
  -v, --v                            Log at INFO level

Use "filebeat otel [command] --help" for more information about a command.
```

(cherry picked from commit f1267b8)

# Conflicts:
#	libbeat/otelbeat/beatconverter/beatconverter_test.go

* fix conflicts

---------

Co-authored-by: Orestis Floros <[email protected]>

Author: mergify[bot]

libbeat/otelbeat/beatconverter/beatconverter.go

@@ -77,6 +77,12 @@ func (c Converter) Convert(_ context.Context, conf *confmap.Conf) error {
 			switch key {
 			case "elasticsearch":
 				esConfig := config.MustNewConfigFrom(output)
+
+				// ignore elasticsearch output if it is not enabled
+				if !esConfig.Enabled() {
+					continue
+				}
+
 				// we use development logger here as this method is part of dev-only otel command
 				logger, _ := logp.NewDevelopmentLogger("")
 				esOTelConfig, err := elasticsearchtranslate.ToOTelConfig(esConfig, logger)

libbeat/otelbeat/beatconverter/beatconverter_test.go

@@ -64,9 +64,9 @@ exporters:
       mode: bodymap
     compression: gzip
     compression_params:
-      level: 1 
+      level: 1
     auth:
-      authenticator: beatsauth  
+      authenticator: beatsauth
 `
 
 func TestConverter(t *testing.T) {
@@ -211,7 +211,7 @@ exporters:
       max_retries: 3
     user: elastic-cloud
     logs_dynamic_pipeline:
-      enabled: true    
+      enabled: true
     max_conns_per_host: 1
     sending_queue:
       batch:
@@ -230,7 +230,7 @@ exporters:
     compression_params:
       level: 1
     auth:
-      authenticator: beatsauth  
+      authenticator: beatsauth
 receivers:
   filebeatreceiver:
     filebeat:
@@ -326,6 +326,43 @@ service:
 		compareAndAssert(t, expOutput, input)
 
 	})
+
+	t.Run("test elasticsearch exporter with enabled false", func(t *testing.T) {
+		var supportedInput = `
+receivers:
+  filebeatreceiver:
+    output:
+      elasticsearch:
+        enabled: false
+        hosts: ["https://localhost:9200"]
+service:
+  pipelines:
+    logs:
+      receivers:
+        - "filebeatreceiver"
+`
+
+		var expectedOutput = `
+receivers:
+  filebeatreceiver:
+    output:
+      otelconsumer: null
+service:
+  telemetry:
+    metrics:
+      level: none
+  pipelines:
+    logs:
+      receivers:
+        - filebeatreceiver
+`
+		input := newFromYamlString(t, supportedInput)
+		err := c.Convert(context.Background(), input)
+		require.NoError(t, err, "error converting beats elasticsearch-output config")
+
+		expOutput := newFromYamlString(t, expectedOutput)
+		compareAndAssert(t, expOutput, input)
+	})
 }
 
 func TestLogLevel(t *testing.T) {
@@ -418,25 +455,25 @@ service:
   pipelines:
     logs:
       receivers:
-        - filebeatreceiver  
+        - filebeatreceiver
 `
 
 	commonOTelCfg := `
 extensions:
   beatsauth:
     idle_connection_timeout: 3s
     proxy_disable: false
-    timeout: 1m30s  
+    timeout: 1m30s
 receivers:
   filebeatreceiver:
     output:
-      otelconsumer: null    
+      otelconsumer: null
 exporters:
   elasticsearch:
     endpoints:
       - http://localhost:9200
     logs_dynamic_pipeline:
-      enabled: true      
+      enabled: true
     retry:
       enabled: true
       initial_interval: 1s
@@ -446,7 +483,7 @@ exporters:
     password: changeme
     user: elastic
     mapping:
-      mode: bodymap 
+      mode: bodymap
     compression: gzip
     compression_params:
       level: 1
@@ -473,7 +510,7 @@ service:
       exporters:
         - elasticsearch
       receivers:
-        - filebeatreceiver      
+        - filebeatreceiver
 `
 
 	tests := []struct {
@@ -490,7 +527,7 @@ receivers:
         events: 3200
         flush:
           min_events: 1600
-          timeout: 10s            
+          timeout: 10s
 extensions:
   beatsauth:
     idle_connection_timeout: 3s
@@ -502,7 +539,7 @@ exporters:
         max_size: 1600
       num_consumers: 1
       queue_size: 3200
-    max_conns_per_host: 1      
+    max_conns_per_host: 1
  `,
 		},
 		{
@@ -515,7 +552,7 @@ receivers:
         events: 12800
         flush:
           min_events: 1600
-          timeout: 5s           
+          timeout: 5s
 extensions:
   beatsauth:
     idle_connection_timeout: 15s
@@ -527,7 +564,7 @@ exporters:
         max_size: 1600
       num_consumers: 4
       queue_size: 12800
-    max_conns_per_host: 4      
+    max_conns_per_host: 4
 `,
 		},
 		{
@@ -540,7 +577,7 @@ receivers:
         events: 3200
         flush:
           min_events: 1600
-          timeout: 20s          
+          timeout: 20s
 extensions:
   beatsauth:
     idle_connection_timeout: 1s
@@ -555,7 +592,7 @@ exporters:
     max_conns_per_host: 1
     retry:
       initial_interval: 5s
-      max_interval: 5m0s       
+      max_interval: 5m0s
 `,
 		},
 		{
@@ -568,7 +605,7 @@ receivers:
         events: 4100
         flush:
           min_events: 2050
-          timeout: 1s          
+          timeout: 1s
 extensions:
   beatsauth:
     idle_connection_timeout: 1m0s
@@ -583,7 +620,7 @@ exporters:
     max_conns_per_host: 1
     retry:
       initial_interval: 1s
-      max_interval: 1m0s    
+      max_interval: 1m0s
 `}}
 
 	commonOTeMap := newFromYamlString(t, commonOTelCfg)