metricset.name: process is not generated for metricbeat run through docker.

[amolnater-qasource]

Kibana Build details:

VERSION: 8.16.0-BC1
BUILD: 79314
COMMIT: 5575428dd3aef69366cddb4ccf07a2a26d30ce48

Artifact: docker.elastic.co/staging/metricbeat:8.16.0-e8d5928a

Host OS: Rhel 9 and Ubuntu 24

Preconditions:

1. 8.16.0-BC1 Kibana cloud environment should be available.
2. Docker should be setup.

Steps to reproduce:

1. Run below command

sudo docker run --label co.elastic.metrics/module=system \
--mount type=bind,source=/proc,target=/hostfs/proc,readonly \
--mount type=bind,source=/sys/fs/cgroup,target=/hostfs/sys/fs/cgroup,readonly \
--mount type=bind,source=/,target=/hostfs,readonly \
--mount type=bind,source=/var/run/dbus/system_bus_socket,target=/hostfs/var/run/dbus/system_bus_socket,readonly \
--env DBUS_SYSTEM_BUS_ADDRESS='unix:path=/hostfs/var/run/dbus/system_bus_socket' \
--net=host --cgroupns=host docker.elastic.co/staging/metricbeat:8.16.0-e8d5928a metricbeat -e -E output.elasticsearch.hosts='https://6***************************d.com:443' \
-E output.elasticsearch.username='elastic' \
-E output.elasticsearch.password='******************' \
-d '*'

2. Observe metricbeat data is generated. However, metricset.name: process is not generated under Discover tab.
3. Now run below command:

sudo docker run --label co.elastic.metrics/module=system \
--mount type=bind,source=/proc,target=/hostfs/proc,readonly \
--mount type=bind,source=/sys/fs/cgroup,target=/hostfs/sys/fs/cgroup,readonly \
--mount type=bind,source=/,target=/hostfs,readonly \
--mount type=bind,source=/var/run/dbus/system_bus_socket,target=/hostfs/var/run/dbus/system_bus_socket,readonly \
--env DBUS_SYSTEM_BUS_ADDRESS='unix:path=/hostfs/var/run/dbus/system_bus_socket' \
--net=host docker.elastic.co/staging/metricbeat:8.16.0-e8d5928a metricbeat -e -E output.elasticsearch.hosts='https://6***************************d.com:443' \
-E output.elasticsearch.username='elastic' \
-E output.elasticsearch.password='******************' \
-d '*'

4. Still observe metricset.name: process is not generated under Discover tab.

Logs:
docker-metricbeat.txt

Expected Result:
metricset.name: process should be generated for metricbeat run through docker.

Screen Recording:

Discover.-.Elastic.-.Google.Chrome.2024-10-23.15-01-24.mp4

Feature:
#39900

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[amolnater-qasource]

@muskangulati-qasource Please review.

[muskangulati-qasource]

Secondary review is Done for this ticket!

[pierrehilbert]

Hey @amolnater-qasource could you please confirm this is working with 8.15.3?

[amolnater-qasource]

Hi @pierrehilbert

Thank you for looking into this issue.
We have revalidated this with docker.elastic.co/beats/metricbeat:8.15.3 and found it reproducible there too.

• metricset.name: process is not generated for metricbeat run through docker.

Further for confirmation we have tested the same with 8.15.2, 8.15.1 and 8.15.0 and had below observations:

• 8.15.2: Reproducible.
• 8.15.1: Not Reproducible
• 8.15.0: Not Reproducible

Screen Recording:

Discover.-.Elastic.-.Google.Chrome.2024-10-23.16-39-16.mp4

Please let us know if anything else is required from our end.
Thanks!!

[pierrehilbert]

I'm looking at changes from 8.15.1 to 8.15.2 but didn't find anything for now.
Maybe @cmacknz would have an idea

[pierrehilbert]

@amolnater-qasource what version of the system integration are you using?
EDIT: forgot what I asked, you are using same stack for every agent you tested.

[cmacknz]

Agree, none of the commits in v8.15.1...v8.15.2 look like they should be related.

[amolnater-qasource]

Hi @pierrehilbert

8.16.0 metricbeat was tested on the same 8.16.0 BC1 kibana cloud environment.

For 8.15.x versions we used the same 8.15.3 kibana cloud environment.
System integration v1.61.1 is available on 8.15.3.

Build details:

VERSION: 8.15.3
BUILD: 76711
COMMIT: 3933429968aafb1ba31319fc38649d0f974044bf

Please let us know if anything else is required from our end.
Thanks!!

[pierrehilbert]

Hey Amol,
Are you facing the same with BC2?

[amolnater-qasource]

Hi @pierrehilbert

We have revalidated this issue on rhel9 with BC2 artifact: docker.elastic.co/staging/metricbeat:8.16.0-66e9e678 and found it still reproducible:

Screen Recording:

Discover.-.Elastic.-.Google.Chrome.2024-10-29.18-53-06.mp4

Please let us know if anything else is required from our end.
Thanks!