Not able to update GEM_PATH Environment variable

[RamHaridas]

Describe the bug

I have installed gems using fluent-gem install --install-dir to a custom path /opt/additionalgems.
I am updating the GEM_PATH environment variable with this path, but instead of loading gems installed in this directory, fluentd is still picking up gems from the default path.

To Reproduce

Install any gem to a custom directory -- fluent-gem install --isntall-dir /opt/additinalgems fluent-plugin-datadog -v 0.14.4

Update the GEM_PATH environment variable with customa path GEM_PATH=/opt/additionalgems:/path/to/default/gems

Run fluentd with a configuration that uses this gem, in our example @type datadog

/usr/sbin/fluentd -c /path/to/fluent.conf

This gives an error that datadog not found.

Expected behavior

Ideally it is expected that fluentd loads the gems installed in the custom directory as well, but in this case it is failing to do so. Regardless of the path addition, it still loads the gems from the default path only.

Your Environment

- Fluentd version: 1.19.0
- Package version: 6.0.0 
- Operating system: Rocky Linux 8
- Kernel version: 5

Your Configuration

<source>
      @type exec
      tag test.echo
      command echo '{"message":"Hello there, from fluentd!"}'
      run_interval 10s
      keys none
    </source>

    <match test.echo>
      @type datadog
      @id awesome_agent
      api_key YOUR_DATADOG_API_KEY
      service fluentd
      dd_source fluentd
      dd_tags env:dev
    </match>

Your Error Log

Fluent config error is thrown with message datadog not found

Additional context

Root cause of this problem is identified. It is the problem with the wrapper script at /usr/sbin/fluentd. The wrapper hard-codes the value for environment variable GEM_PATH to default location. This basically overrides the value set by user and causing fluentd to load only gems saved at default path.

Below given is the wrapper script:

#!/opt/fluent/bin/ruby
ENV["GEM_HOME"]="/opt/fluent/lib/ruby/gems/3.4.0/"
ENV["GEM_PATH"]="/opt/fluent/lib/ruby/gems/3.4.0/"   # ---> HARD-CODED
ENV["FLUENT_CONF"]="/etc/fluent/fluentd.conf"
ENV["FLUENT_CONF_INCLUDE_DIR"]="/etc/fluent/conf.d"
ENV["FLUENT_PLUGIN"]="/etc/fluent/plugin"
ENV["FLUENT_SOCKET"]="/var/run/fluent/fluentd.sock"
if ARGV.include?("--version")
  require "/opt/fluent/share/config"
  Dir.glob("/opt/fluent/lib/ruby/**/gems/**/fluent/version.rb").each do |v|
    require v.delete_suffix(".rb")
  end
  puts "fluent-package #{PACKAGE_VERSION} fluentd #{Fluent::VERSION} (#{FLUENTD_REVISION})"
  exit 0
end

if RUBY_PLATFORM =~ /linux/
  prevent_duplicate_launch = system("systemctl", "is-active", "fluentd", out: IO::NULL)
  if prevent_duplicate_launch
    if ["-c", "--config", "--dry-run", "-h", "--help", "--show-plugin-config"].none? {|allowing_opt| ARGV.include? allowing_opt}
      puts("Error: Can't start duplicate Fluentd instance with the default config.")
      if ARGV.include?("-v")
        puts("To take the version, please use '--version', not '-v' ('--verbose').")
      end
      puts <<EOS
To start Fluentd, please do one of the following:
(Caution: Please be careful not to start multiple instances with the same config.)
- Stop the Fluentd service 'fluentd'.
- Specify the config path explicitly by '-c' ('--config').
EOS
      exit 2
    end
  end
end

load "/opt/fluent/bin/fluentd"

When i run /opt/fluent/bin/fluent-gem list, I can see the datadog gem in the list, because it reads the GEM_PATH env set by the user, which is not the case with /usr/sbin/fluentd.

Is there a workaround to overcome this without having to edit the wrapper script or needing to replace it with /opt/fluent/bin/fluentd binary.
Or is there a parameter that works as a replacement for GEM_PATH environment variable?