Add support for injecting configuration data into the transform definitions

[kcreddy]

Currently integration configuration data, for example interval, cannot be injected into the transform definition, confirmed here.

Making the transform retention user-configurable is often asked before, for example here. Some integrations have a workaround by injecting variables into input config, then into the ingest pipeline, and then into the transform through a field. This workaround is less efficient. It would be nice to control it better by directly injecting the configuration data into the transforms. Also a direct option to control the transform would have a better user experience as well.

By allowing user configuration data injected into the transform, users can control the transform retention efficiently.

Example: Based on interval value,

retention_policy:
  time:
    field: "@timestamp"
    max_age: {{ interval }}

[cpascale43]

+1

[cpascale43]

Hey @kpollich @nimarezainia - are there any plans to do this already? Will route it through the dependency board flow if not

[nimarezainia]

@cpascale43 I don't think we have had this request before.