Support Env Var expansion in toml config files

[rocktavious]

• Which Faktory package and version? Enterprise 1.8
• Which Faktory worker package and version? Enterprise 1.8

Are you using an old version? Just off by 1 version
Have you checked the changelogs to see if your issue has been fixed in a later version? Yes

I'm trying to setup https://github.com/contribsys/faktory/wiki/Ent-Metrics in an kubernetes cluster with a datadog agent setup.

Normally for statsd you configure the downward API to target datadog pod like this

          env:
            - name: DD_AGENT_HOST
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP

This means the environment variable $DD_AGENT_HOST will be filled with the IP address that faktory should send the statsd metrics to. The problem is that to configure faktory to send statsd metrics you have to pass a toml file which it appears does not support environment variable expansion.

Here is the toml file I want to use

[statsd]
location = "${DD_AGENT_HOST}:8125"
tags = ["env:production"]
queueLatency = ["default"]

But unfortunately it appears the variable is not expanded because of these logs lines

opslevel-faktory-0 server I 2024-06-05T02:15:11.997Z Web server now listening at :7420
opslevel-faktory-0 server I 2024-06-05T02:15:11.997Z Sending statsd metrics to ${DD_AGENT_HOST}:8125 with namespace faktory

Is there a way to support this?

[mperham]

BurntSushi/toml#80

[mperham]

Not easily. You could generate the .toml or add a known DNS entry in /etc/hosts which points to the DD host for the container as part of your deploy.

[rocktavious]

Thanks @mperham - I've setup a internal service that creates a cluster DNS entry datadog-agent.datadog.svc.cluster.local and that seems to work. Would be nice to have a go template or env expansion applied to the TOML file at runtime before being loaded - though I understand the constraints.