Skip to content

Data Streams live messages #20512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

piochelepiotr
Copy link
Contributor

@piochelepiotr piochelepiotr commented Jun 13, 2025

What does this PR do?

The integration reads from a config modified by the agent in this PR and retrieves messages from Kafka. It then logs the messages, for the Datadog UI to pick them up.

Motivation

We are working on a new feature called "Live Messages" for Data Streams, for Kafka.
The feature will enable users to retrieve messages from Kafka on-demand, and send them as logs to Datadog.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Copy link

codecov bot commented Jun 30, 2025

Codecov Report

Attention: Patch coverage is 72.07792% with 43 lines in your changes missing coverage. Please review.

Project coverage is 88.56%. Comparing base (2df1b1d) to head (28b5bc5).

Additional details and impacted files
Flag Coverage Δ
activemq ?
hive ?
hivemq ?
hudi ?
ignite ?
jboss_wildfly ?
kafka_consumer 89.38% <72.07%> (-1.67%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@piochelepiotr piochelepiotr force-pushed the piotr-wolski/live-messages branch 3 times, most recently from 9f8dd71 to 4b1cde2 Compare July 3, 2025 20:11
@piochelepiotr piochelepiotr force-pushed the piotr-wolski/live-messages branch 3 times, most recently from 0c51a3d to b8ad8f3 Compare July 3, 2025 20:20
@piochelepiotr piochelepiotr force-pushed the piotr-wolski/live-messages branch from b8ad8f3 to 28b5bc5 Compare July 3, 2025 20:25
@piochelepiotr piochelepiotr marked this pull request as ready for review July 3, 2025 20:25
@piochelepiotr piochelepiotr requested a review from a team as a code owner July 3, 2025 20:25
Comment on lines +418 to +419
# if cluster != cluster_id:
# continue

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to check for this?

@piochelepiotr piochelepiotr requested a review from iliakur July 7, 2025 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants