Kafka broker inline logger configuration causing TRACE and DEBUG logging

[jslusher]

I'm using Strimzi Kafka Operator version 0.44.0 with Kafka 3.7.1
I have the following set for inline logging:

spec:
  kafka:
    logging:
      type: inline
      loggers:
        kafka.root.logger.level: "INFO"
        log4j.logger.kafka.server.ZkConfigManager: "ERROR"
        log4j.logger.kafka.common.ZkNodeChangeNotificationListener: "ERROR"
        log4j.logger.io.strimzi.kafka.quotas.VolumeSource: "ERROR"
        log4j.logger.io.strimzi.kafka.quotas.throttle.PolicyBasedThrottle: "ERROR"

Originally I had just kafka.root.logger.level set, but I'm trying to reduce log output of some of the chattier loggers.

The settings I added seems to have at least filtered out the logs I specified. My understanding is that anything I don't specify defaults to the root logger level, INFO, but I'm seeing TRACE and DEBUG logs after adding the other loggers. It feels similar to a bug I ran into last year in Kafka Connect using an older version of the operator.

Here are some of the logs I'm seeing after setting that configuration. I was not seeing these before:

2024-12-06 16:46:55,235 DEBUG [Controller id=0] Topics not in preferred replica for broker 1 HashMap() (kafka.controller.KafkaController) [controller-event-thread]
2024-12-06 16:46:55,235 TRACE [Controller id=0] Leader imbalance ratio for broker 1 is 0.0 (kafka.controller.KafkaController) [controller-event-thread]
2024-12-06 16:46:55,235 DEBUG [Controller id=0] Topics not in preferred replica for broker 2 HashMap() (kafka.controller.KafkaController) [controller-event-thread]
2024-12-06 16:46:55,235 TRACE [Controller id=0] Leader imbalance ratio for broker 2 is 0.0 (kafka.controller.KafkaController) [controller-event-thread]

Is there something I'm doing wrong or is it possible this is a new bug or regression?

[scholzj]

The default logging configuration for Kafka is based on this file: https://github.com/strimzi/strimzi-kafka-operator/blob/main/cluster-operator/src/main/resources/default-logging/KafkaCluster.properties -> so your inline configurations are essentially added to it. As you can see, it has some loggers enabled at TRACE levels. So I think what you see is expected and if you want to avoid it, you should configure all the different loggers.

Please not ethat the default Strimzi configuration is (at least originally - it might have changed over time here and there) based on the Kafka default configuration: https://github.com/apache/kafka/blob/trunk/config/log4j.properties

[jslusher]

Looking a bit closer I see that we have TRACE and DEBUG level logs even in the cluster that currently only has kafka.root.logger.level: INFO set. I think my misunderstanding was that if you set the root logger, it means everything not set defaults to the root logger setting. I was not aware that previously set defaults override this setting. Thanks for the explanation!