Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x](backport #5833) Add filebeat receiver to otel mode #5894

Open
wants to merge 4 commits into
base: 8.x
Choose a base branch
from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Oct 30, 2024

What does this PR do?

Adds filebeat receiver as an otel receiver. This is a re-introduction of code that was added in #5672, that had to be reverted because the import of the beats code base was causing issues with command line parsing. The issues in the beats code base have been fixed, so we are re-introducing this change.

Why is it important?

This is the first beat receiver, and this will allow us to test the feasibility of running beats as otel receivers.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

should be 0 user impact of existing configurations

How to test this PR locally

Use something like the following as otel.yml, and run elastic-agent otel --config otel.yml
This will run the beat receiver, which will produce one event.
The event should be written to the path you specify in the file/no_rotation section.

receivers:
  filebeatreceiver:
    filebeat:
      inputs:
        - type: benchmark
          enabled: true
          count: 1
          message: "The quick brown fox jumped over the lazy dog"
    output:
      otelconsumer:
    logging:
      level: info
      selectors:
        - '*'
    path:
      home: /tmp/<state dir>
exporters:
  file/no_rotation:
    path: /tmp/<output file>
service:
  pipelines:
    logs:
      receivers: [filebeatreceiver]
      exporters: [file/no_rotation]

Related issues

Questions to ask yourself

  • How are we going to support this in production?
  • How are we going to measure its adoption?
  • How are we going to debug this?
  • What are the metrics I should take care of?
  • ...

This is an automatic backport of pull request #5833 done by [Mergify](https://mergify.com).

(cherry picked from commit 89dca1d)

# Conflicts:
#	NOTICE.txt
#	go.mod
#	go.sum
@mergify mergify bot requested a review from a team as a code owner October 30, 2024 21:22
@mergify mergify bot requested review from blakerouse and removed request for a team October 30, 2024 21:22
@mergify mergify bot added the backport label Oct 30, 2024
@mergify mergify bot requested a review from kaanyalti October 30, 2024 21:22
@mergify mergify bot added the conflicts There is a conflict in the backported pull request label Oct 30, 2024
Copy link
Contributor Author

mergify bot commented Oct 30, 2024

Cherry-pick of 89dca1d has failed:

On branch mergify/bp/8.x/pr-5833
Your branch is up to date with 'origin/8.x'.

You are currently cherry-picking commit 89dca1d1d4.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1729630977-Add-filebeat-otel-receiver.yaml
	modified:   dev-tools/notice/overrides.json
	modified:   internal/pkg/otel/components.go
	modified:   testing/integration/otel_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   NOTICE.txt
	both modified:   go.mod
	both modified:   go.sum

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@pierrehilbert pierrehilbert requested review from leehinman and removed request for blakerouse and kaanyalti October 31, 2024 08:31
@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Oct 31, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

Copy link

Copy link
Contributor Author

mergify bot commented Nov 4, 2024

This pull request has not been merged yet. Could you please review and merge it @leehinman? 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport conflicts There is a conflict in the backported pull request Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants