Azure active directory integration for strimzi kafka for authentication and authorization

[Raju-kf]

Hi, I want to implement Azure ad integration with strimzi kafka for authentication and authorization. I had used azure ad end points to configure authentication. But I'm facing issue with those configurations. Havent found any documentation regarding this feature. Please suggest any changes or anything need to be add.

--> Does strimzi kafka supports Authentication and authorization with Azure Ad.?

Kafka.yaml:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 2.8.0
    replicas: 1
    resources:
      requests:
        memory: 1Gi
        cpu: "1"
      limits:
        memory: 1Gi
        cpu: "1"
    listeners:
      - name: extnl
        port: 32244
        type: loadbalancer
        tls: false
        authentication:
          clientId: ce38457b-3905-40df-9a27-70fb1fb2e7ca
          clientSecret:
            key: secret
            secretName: broker-oauth-secret
          type: oauth
          validIssuerUri: https://login.microsoftonline.com/xxxxxxxxxxxxxxxxxxxxxx/v2.0
          jwksEndpointUri: https://login.microsoftonline.com/xxxxxxxxxxxxxxxxxx/discovery/v2.0/key
          userNameClaim: preferred_username
    logging:
      type: inline
      loggers:
        log4j.logger.io.strimzi: "TRACE"
        log4j.logger.kafka: "DEBUG"
        log4j.logger.org.apache.kafka: "DEBUG"
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      log.message.format.version: "2.8"
      inter.broker.protocol.version: "2.8"
    storage:
      type: persistent-claim
      size: 10Gi
      class: gp2
      deleteClaim: false
  zookeeper:
    replicas: 1
    storage:
      type: persistent-claim
      size: 10Gi
      class: gp2
      deleteClaim: false
  entityOperator:
    topicOperator: {}
    userOperator: {}

Kafka pod logs:

2021-12-06 13:18:10,915 INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas) [main]
2021-12-06 13:18:10,916 INFO Awaiting socket connections on 0.0.0.0:32244. (kafka.network.Acceptor) [main]
2021-12-06 13:18:11,518 ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) [main]
org.apache.kafka.common.KafkaException: java.lang.RuntimeException: Failed to fetch public keys needed to validate JWT signatures: 
https://login.microsoftonline.com/xxxxxxxxxxxxxxxxxxxxxxxxx/discovery/v2.0/key
        at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184)
        at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
        at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:107)
        at kafka.network.Processor.<init>(SocketServer.scala:853)
        at kafka.network.SocketServer.newProcessor(SocketServer.scala:442)
        at kafka.network.SocketServer.$anonfun$addDataPlaneProcessors$1(SocketServer.scala:299)
        at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190)
        at kafka.network.SocketServer.addDataPlaneProcessors(SocketServer.scala:297)
        at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1(SocketServer.scala:262)
        at kafka.network.SocketServer.$anonfun$createDataPlaneAcceptorsAndProcessors$1$adapted(SocketServer.scala:259)
        at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
        at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
        at kafka.network.SocketServer.createDataPlaneAcceptorsAndProcessors(SocketServer.scala:259)
        at kafka.network.SocketServer.startup(SocketServer.scala:131)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:285)
        at kafka.Kafka$.main(Kafka.scala:109)
        at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.RuntimeException: Failed to fetch public keys needed to validate JWT signatures: 
https://login.microsoftonline.com/xxxxxxxxxxxxxxxxxxxxxxxxx/discovery/v2.0/key
        at io.strimzi.kafka.oauth.validator.JWTSignatureValidator.fetchKeys(JWTSignatureValidator.java:259)
        at io.strimzi.kafka.oauth.validator.JWTSignatureValidator.<init>(JWTSignatureValidator.java:145)
        at io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler.lambda$configure$0(JaasServerOauthValidatorCallbackHandler.java:282)
        at io.strimzi.kafka.oauth.services.Validators.lambda$get$0(Validators.java:17)
        at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
        at io.strimzi.kafka.oauth.services.Validators.get(Validators.java:17)
        at io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler.configure(JaasServerOauthValidatorCallbackHandler.java:333)
        at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:151)
        ... 17 more
Caused by: io.strimzi.kafka.oauth.common.HttpException: GET request to 
https://login.microsoftonline.com/xxxxxxxxxxxxxxxxxxxxxxxxx/discovery/v2.0/key failed with status 404: Not Found
        at io.strimzi.kafka.oauth.common.HttpUtil.handleResponse(HttpUtil.java:162)
        at io.strimzi.kafka.oauth.common.HttpUtil.request(HttpUtil.java:143)
        at io.strimzi.kafka.oauth.common.HttpUtil.request(HttpUtil.java:86)
        at io.strimzi.kafka.oauth.common.HttpUtil.get(HttpUtil.java:50)
        at io.strimzi.kafka.oauth.validator.JWTSignatureValidator.fetchKeys(JWTSignatureValidator.java:232)
        ... 24 more
2021-12-06 13:18:11,522 INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer) [main]
2021-12-06 13:18:11,524 INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Stopping socket server request processors (kafka.network.SocketServer) [main]
2021-12-06 13:18:11,528 INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Stopped socket server request processors (kafka.network.SocketServer) [main]
2021-12-06 13:18:11,529 DEBUG Shutting down task scheduler. (kafka.utils.KafkaScheduler) [main]
2021-12-06 13:18:11,534 INFO Shutting down. (kafka.log.LogManager) [main]
2021-12-06 13:18:11,535 INFO Shutting down the log cleaner. (kafka.log.LogCleaner) [main]
2021-12-06 13:18:11,535 INFO [kafka-log-cleaner-thread-0]: Shutting down (kafka.log.LogCleaner) [main]
2021-12-06 13:18:11,536 INFO [kafka-log-cleaner-thread-0]: Stopped (kafka.log.LogCleaner) [kafka-log-cleaner-thread-0]
2021-12-06 13:18:11,536 INFO [kafka-log-cleaner-thread-0]: Shutdown completed (kafka.log.LogCleaner) [main]
2021-12-06 13:18:11,542 DEBUG Flushing and closing logs at /var/lib/kafka/data/kafka-log0 (kafka.log.LogManager) [main]
2021-12-06 13:18:11,544 DEBUG Updating recovery points at /var/lib/kafka/data/kafka-log0 (kafka.log.LogManager) [main]
2021-12-06 13:18:11,550 DEBUG Updating log start offsets at /var/lib/kafka/data/kafka-log0 (kafka.log.LogManager) [main]
2021-12-06 13:18:11,554 DEBUG Writing clean shutdown marker at /var/lib/kafka/data/kafka-log0 (kafka.log.LogManager) [main]
2021-12-06 13:18:11,556 INFO Shutdown complete. (kafka.log.LogManager) [main]
2021-12-06 13:18:11,556 INFO [feature-zk-node-event-process-thread]: Shutting down (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread) [main]
2021-12-06 13:18:11,557 INFO [feature-zk-node-event-process-thread]: Stopped (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread) [feature-zk-node-event-process-thread]
2021-12-06 13:18:11,557 INFO [feature-zk-node-event-process-thread]: Shutdown completed (kafka.server.FinalizedFeatureChangeListener$ChangeNotificationProcessorThread) [main]
2021-12-06 13:18:11,558 INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient) [main]
2021-12-06 13:18:11,558 DEBUG Shutting down task scheduler. (kafka.utils.KafkaScheduler) [main]
2021-12-06 13:18:11,612 INFO channel is told closing (org.apache.zookeeper.ClientCnxnSocketNetty) [main]
2021-12-06 13:18:11,614 INFO channel is disconnected: [id: 0xe13b70ed, L:/10.192.10.227:57098 ! R:my-cluster-zookeeper-client/172.20.159.144:2181] (org.apache.zookeeper.ClientCnxnSocketNetty) [nioEventLoopGroup-2-1]
2021-12-06 13:18:11,614 INFO channel is told closing (org.apache.zookeeper.ClientCnxnSocketNetty) [nioEventLoopGroup-2-1]
2021-12-06 13:18:11,615 DEBUG [ZooKeeperClient Kafka server] Received event: WatchedEvent state:Closed type:None path:null (kafka.zookeeper.ZooKeeperClient) [main-EventThread]
2021-12-06 13:18:11,615 INFO Session: 0x100adec69780058 closed (org.apache.zookeeper.ZooKeeper) [main]
2021-12-06 13:18:11,615 INFO EventThread shut down for session: 0x100adec69780058 (org.apache.zookeeper.ClientCnxn) [main-EventThread]
2021-12-06 13:18:11,616 INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient) [main]
2021-12-06 13:18:11,617 INFO [ThrottledChannelReaper-Fetch]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:11,649 TRACE Polling kafka.server:type=SessionExpireListener,name=SessionState (io.strimzi.kafka.agent.KafkaAgent) [KafkaAgentPoller]
2021-12-06 13:18:11,649 TRACE Polling kafka.server:type=KafkaServer,name=BrokerState (io.strimzi.kafka.agent.KafkaAgent) [KafkaAgentPoller]
2021-12-06 13:18:11,740 INFO [ThrottledChannelReaper-Fetch]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Fetch]
2021-12-06 13:18:11,740 INFO [ThrottledChannelReaper-Fetch]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:11,740 INFO [ThrottledChannelReaper-Produce]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:12,649 TRACE Polling kafka.server:type=SessionExpireListener,name=SessionState (io.strimzi.kafka.agent.KafkaAgent) [KafkaAgentPoller]
2021-12-06 13:18:12,650 TRACE Polling kafka.server:type=KafkaServer,name=BrokerState (io.strimzi.kafka.agent.KafkaAgent) [KafkaAgentPoller]
2021-12-06 13:18:12,740 INFO [ThrottledChannelReaper-Produce]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Produce]
2021-12-06 13:18:12,740 INFO [ThrottledChannelReaper-Produce]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:12,740 INFO [ThrottledChannelReaper-Request]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:12,742 INFO [ThrottledChannelReaper-Request]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-Request]
2021-12-06 13:18:12,742 INFO [ThrottledChannelReaper-Request]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:12,742 INFO [ThrottledChannelReaper-ControllerMutation]: Shutting down (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:12,809 INFO [ThrottledChannelReaper-ControllerMutation]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [ThrottledChannelReaper-ControllerMutation]
2021-12-06 13:18:12,809 INFO [ThrottledChannelReaper-ControllerMutation]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper) [main]
2021-12-06 13:18:12,810 INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Shutting down socket server (kafka.network.SocketServer) [main]
2021-12-06 13:18:12,908 INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Shutdown completed (kafka.network.SocketServer) [main]
2021-12-06 13:18:12,909 INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics) [main]
2021-12-06 13:18:12,910 INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics) [main]
2021-12-06 13:18:12,910 INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics) [main]
2021-12-06 13:18:12,913 INFO Broker and topic stats closed (kafka.server.BrokerTopicStats) [main]
2021-12-06 13:18:12,917 INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser) [main]
2021-12-06 13:18:12,918 INFO [KafkaServer id=0] shut down completed (kafka.server.KafkaServer) [main]
2021-12-06 13:18:12,918 ERROR Exiting Kafka. (kafka.Kafka$) [main]
2021-12-06 13:18:12,918 INFO [KafkaServer id=0] shutting down (kafka.server.KafkaServer) [kafka-shutdown-hook]

[rsavithry]

Hello @Raju-kf , did you get any solution for this issue?

[einarmo]

I'll be a good citizen. For anyone running into this it's /discovery/v2.0/keys, not /discovery/v2.0/key.