Skip to content

[newrelic-logging] fluent bit metric throwing 400 status code #1743

@sumitgarg44

Description

@sumitgarg44

Bug description
Fluent bit internal metrics throwing 400 Bad Request when trying to send metrics to newrelic as per following default non-customizable config. It looks like, prometheus_scrape output in text format and prometheus_remote_write expects input in protobuf format which raises the conflict.

    [INPUT]
        name prometheus_scrape
        Alias fb-metrics-collector
        host 127.0.0.1
        port 2020
        tag fb_metrics
        metrics_path /api/v2/metrics/prometheus
        scrape_interval 60s

    [OUTPUT]
        Name                 prometheus_remote_write
        Match                fb_metrics
        Alias                fb-metrics-forwarder
        Host                 ${METRICS_HOST}
        Port                 443
        Uri                  /prometheus/v1/write?prometheus_server=${CLUSTER_NAME}
        Header               Authorization Bearer ${LICENSE_KEY}
        Tls                  On
        Tls.verify           Off
        add_label            app fluent-bit
        add_label            source kubernetes
        add_label            pod_name ${HOSTNAME}
        add_label            node_name ${NODE_NAME}
        add_label            cluster_name Stepladder Staging
        add_label            namespace newrelic
        add_label            daemonset_name newrelic-bundle-newrelic-logging

Version of Helm and Kubernetes
k8s_version: 1.30.13
helm_version: 3.8.1

Which chart?
nri-bundle 6.0.0

What happened?
fluent bit throwing 400 status code when trying to push internal metrics to newrelic.

What you expected to happen?

It should be able to submit metrics to newrelic, and data type conflict between INPUT and OUTPUT should be resolved.

How to reproduce it?

Should be able to reproduce it with default config.

Anything else we need to know?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugCategorizes issue or PR as related to a bug.triage/pendingIssue or PR is pending for triage and prioritization.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions