Skip to content

Fluentd fails to load separate gems with same @type name #5147

@RamHaridas

Description

@RamHaridas

Describe the bug

I have installed two splunk gems, namely, fluent-plugin-splunk-enterprise and fluent-plugin-splunk-hec. Both the gems (even though 2 separate gems) have types with same name splunk_hec. Even though the type names are same, they have different parameters, that makes them distinct.
But when I use @type splunk_hec in my fluentd configuration with the parameters of fluent-plugin-splunk-enterprise, it fails to recognize them and throws below error:

2025-11-06 17:05:28 +0000 [error]: config error file="/etc/fluent/fluentd.conf" error_class=Fluent::ConfigError error="'hec_token' parameter is required"

It asks for hec_token, which is basically a parameter of fluent-plugin-splunk-hec and not fluent-plugin-splunk-enterprise.

To Reproduce

This bug can be reproduced by simply installing both these gems:

  • fluent-gem install fluent-plugin-splunk-hec
  • fluent-gem install fluent-plugin-splunk-enterprise

Fluentd will automatically register the @type name from first gem and ignore the second one. (which is happening in my case)

Expected behavior

Now, I do understand that this is a known behavior of fluentd and it has been like that for some time. So, this understanding brings me to a question that is there a workaround for this without having to remove any one of the 2 gems?

Is there a way that we can install both gems and use them in our configuration without conflicting with each other??

Your Environment

- Fluentd version: 1.19.0
- Package version: 6.0.0
- Operating system: Rocky 8
- Kernel version: 5

Your Configuration

<match splunk.**>
  @type splunk_hec
  protocol http
  host "example.com"
  port 8088
  token 00000000-0000-0000-0000-000000000000
  sourcetype "_json"
  <buffer>
    @type memory
    flush_interval 10s
    retry_max_times 2
    retry_wait 1s
  </buffer>
</match>

Your Error Log

2025-11-06 17:05:28 +0000 [error]: config error in:
<match splunk.**>
  @type splunk_hec
  protocol http
  host "example.com"
  port 8088
  token 00000000-0000-0000-0000-000000000000
  sourcetype "_json"
  <buffer>
    @type memory
    flush_interval 10s
    retry_max_times 2
    retry_wait 1s
  </buffer>
</match>

2025-11-06 17:05:28 +0000 [error]: config error file="/etc/fluent/fluentd.conf" error_class=Fluent::ConfigError error="'hec_token' parameter is required"

Additional context

I am well aware of the workaround by forking the fluent-plugin-splunk-enterprise and renaming the type, and then building the gem. But this workaround is not feasible and convenient for my use case.
I am much interested in a solution where fluentd handles this internally or we handle it internally without having to make modifications in the gem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementFeature request or improve operations

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions