[QUESTION] kafka super user with tls-external authentication denied authorisation in Kafka strimzi cluster

[OLAMIDE100]

GENERAL SETUP

1. Both CLUSTER CA and Client CA where created externally with the right secret annotation and label supplied to the cluster

2. The User certificate and key where created external and signed with the client CA, all stored in pkcs12 trust store and key store

KAFKA YAML

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  clusterCa:
    generateCertificateAuthority: false
  clientsCa:
    generateCertificateAuthority: false
  kafka:
    authorization:
      type: simple
      superUsers:
        - CN=super-user,OU=Dataops,O=*******,L=***********,C=EE
    version: 3.9.0
    metadataVersion: 3.9-IV0
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
        authentication:
          type: scram-sha-512
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external2
        port: 9094
        type: loadbalancer
        tls: true
        configuration:
          bootstrap:
            loadBalancerIP: **********
          brokers:
          - broker: 0
            loadBalancerIP: ************
        authentication:
          type: tls
    template:
      pod:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
                - matchExpressions:
                  - key: cloud.google.com/gke-nodepool
                    operator: In
                    values:
                    - kafka-pool
        tolerations:
          - key: "app.kubernetes.io/instance"
            operator: "Equal"
            value: "kafka"
            effect: "NoSchedule"
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
      default.replication.factor: 1
      min.insync.replicas: 1
  entityOperator:
    topicOperator: {}
    userOperator: {}
    

USER YAML

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: super-user
  labels:
    strimzi.io/cluster: my-cluster
spec:
  authentication:
    type: tls-external

SECRET CREATEION

kubectl create secret generic my-cluster-cluster-ca-cert  -n kafka \
  --from-file=ca.crt=/Users/adewale.adesoba/Documents/company_bitbucket/Private_key_infrastructure/kafka-cluster/certificate_authority/ca.crt \
  --from-file=ca.p12=/Users/adewale.adesoba/Documents/company_bitbucket/Private_key_infrastructure/kafka-cluster/certificate_authority/ca.p12 \
  --from-literal=ca.password=PWOC0LxFiRzN

kubectl create secret generic my-cluster-cluster-ca -n kafka \
  --from-file=ca.key=/Users/adewale.adesoba/Documents/company_bitbucket/Private_key_infrastructure/kafka-cluster/certificate_authority/ca.key


kubectl create secret generic my-cluster-clients-ca-cert -n kafka \
  --from-file=ca.crt=/Users/adewale.adesoba/Documents/company_bitbucket/Private_key_infrastructure/kafka-cluster/client_authority/ca.crt \
  --from-file=ca.p12=/Users/adewale.adesoba/Documents/company_bitbucket/Private_key_infrastructure/kafka-cluster/client_authority/ca.p12 \
  --from-literal=ca.password=OeutymenuBFj


kubectl create secret generic my-cluster-clients-ca -n kafka \
  --from-file=ca.key=/Users/adewale.adesoba/Documents/company_bitbucket/Private_key_infrastructure/kafka-cluster/client_authority/ca.key

kubectl label secret my-cluster-cluster-ca  strimzi.io/kind=Kafka strimzi.io/cluster=my-cluster -n kafka 
kubectl label secret my-cluster-cluster-ca-cert  strimzi.io/kind=Kafka strimzi.io/cluster=my-cluster -n kafka 
kubectl label secret my-cluster-clients-ca-  strimzi.io/kind=Kafka strimzi.io/cluster=my-cluster -n kafka 
kubectl label secret my-cluster-clients-ca-cert  strimzi.io/kind=Kafka strimzi.io/cluster=my-cluster -n kafka 

kubectl annotate secret my-cluster-cluster-ca strimzi.io/ca-key-generation=0 -n kafka
kubectl annotate secret my-cluster-clients-ca strimzi.io/ca-key-generation=0 -n kafka

kubectl annotate secret my-cluster-clients-ca-cert strimzi.io/ca-cert-generation=0 -n kafka
 kubectl annotate secret my-cluster-cluster-ca-cert strimzi.io/ca-cert-generation=0 -n kafka

ERROR AT THE BROKER POD

textPayload: "2025-01-16 13:13:20,802 INFO Principal = User:1.2.840.113549.1.9.1=#161761646577616c652e616465736f6261406e6162752e6565,CN=super-user,OU=Dataops,O=**********,L=**************,C=EE is Denied operation = DESCRIBE from host = 10.164.15.214 on resource = Topic:LITERAL:kemi for request = Metadata with resourceRefCount = 1 based on rule DefaultDeny (kafka.authorizer.logger) [data-plane-kafka-request-handler-1]"

[scholzj]

I think that for a start you would need to fix the formatting to make the YAMLs readable.

[OLAMIDE100]

done

[scholzj]

I think your certificate subject is not as you expected it ... based onthe authorization error, it looks like the username is 1.2.840.113549.1.9.1=#161761646577616c652e616465736f6261406e6162752e6565,CN=super-user,OU=Dataops,O=**********,L=**************,C=EE

So that is what you need to use in the Kafka CR:

    authorization:
      type: simple
      superUsers:
        - 1.2.840.113549.1.9.1=#161761646577616c652e616465736f6261406e6162752e6565,CN=super-user,OU=Dataops,O=**********,L=**************,C=EE

Also, you do not need the KafkaUser resource in this case, it is meaning less because even if you add anything to it, it will have no relation to the user fdrom your certificate (as it would need to have the subject only as CN=super-user to match the KafkaUser resource)

[OLAMIDE100]

wow, thanks so much, I have been troubleshooting for over two days, so I have some quick questions

1. where are the figures before the CN=super-user coming from because I cant see it in the certificate

2. so for this ((as it would need to have the subject only as CN=super-user to match the KafkaUser resource) does it mean that even if I add some set of authorization to this user in the user.yaml file (assume it is not a super user) those acl wont be effective and how to solve such an issue in case in the future

[OLAMIDE100]

for the question 1, i got the answer already incase of future reference here

Understanding the Prefix
OID (Object Identifier):
1.2.840.113549.1.9.1 is an Object Identifier that corresponds to the X.509 attribute type for an email address (commonly referred to as emailAddress in certificates).

Value Encoding:

#1617... indicates that the value is encoded in ASN.1 (Abstract Syntax Notation One).
The #16 specifies that the data is of type IA5String (a string type commonly used for email addresses).
1761646577616c652e616465736f6261406e6162752e6565 is the hexadecimal representation of the email address.
Decoded, it corresponds to:
email address

[scholzj]

where are the figures before the CN=super-user coming from because I cant see it in the certificate

From your certificate - but you would need to check where exactly (I assume you cannot share it, but I guess you can for example pass it through OpenSSL to see the details).

so for this ((as it would need to have the subject only as CN=super-user to match the KafkaUser resource) does it mean that even if I add some set of authorization to this user in the user.yaml file (assume it is not a super user) those acl wont be effective and how to solve such an issue in case in the future

Correct. If you added some ACLs there to the KafkaUser resource, they will be condifugred for user CN=super-user. So the certificate subject would need to correspond to it. Or you would need to use something like this https://github.com/scholzj/custom-strimzi-principal-builder to make the certificate fit the required username.

[OLAMIDE100]

Thanks so much, it is better to streamline the users certificate to just the CN, to save me a lot of trouble, I will look into the custom builder in the future