The in_cloudwatch_log doesn't pick up the newly created logstream after fluentd initialized

[rendyfebry]

Problem

When I use the use_log_group_name_prefix and use_log_stream_name_prefix, this plugin works as expected and is able to fetch logs from all satisfied log_group and log_stream. But after the if there's a new log_stream created during fluentd run time, the plugin won't detect this new log_stream, therefor the logs not being ingested.

...

Steps to replicate

1. Run Fluentd with this configuration

<source>
  @type cloudwatch_logs

  log_group_name /group-a
  use_log_group_name_prefix true
  log_stream_name app
  use_log_stream_name_prefix true
  fetch_interval 5
</source>

2. During the runtime, create a new log_stream that still satisfy the above configuration
3. You'll notice the plugin won't detect this new stream

Expected Behavior or What you need to ask

It should detect new log_stream as long as it satisfy the configuration.

...

Using Fluentd and CloudWatchLogs plugin versions

• OS version:
• Bare Metal or within Docker or Kubernetes or others? Docker
• Fluentd v0.12 or v0.14/v1.0
  • fluent/fluentd:v1.13-debian-1
• Dependent gem versions

2022-01-21 11:09:44 +0000 [info]: starting fluentd-1.14.4 pid=7 ruby="2.6.8"
2022-01-21 11:09:44 +0000 [info]: spawn command to main:  cmdline=["/usr/local/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/local/bundle/bin/fluentd", "-c", "/fluentd/etc/fluent.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]