scram-sha-512 authentication

[sleepdan]

Hello everyone! I want to set up authentication using scram-sha-512. Here is my config:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    replicas: 1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: nodeport
        tls: false
        authentication:
          type: scram-sha-512
    authorization:
      type: simple
    storage:
      type: persistent-claim
      size: 1Gi
      deleteClaim: false
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
  zookeeper:
    replicas: 1
    storage:
      type: persistent-claim
      size: 1Gi
      deleteClaim: false
  entityOperator:
    topicOperator: {}
    userOperator: {}

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: user1
  labels:
    strimzi.io/cluster: acl-test
spec:
  authentication:
    type: scram-sha-512
  authorization:
    type: simple
    acls:
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Read
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Write
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Describe
      - resource:
          type: group
          name: my-group
          patternType: literal
        operation: Read

The documentation says that "When the user is created by the User Operator, it creates a new secret with the same name as the KafkaUser resource.", But in my case Secrets is not created:

#kubectl -n acl-test get kafkausers.kafka.strimzi.io 
NAME    CLUSTER    AUTHENTICATION   AUTHORIZATION   READY
user1   acl-test   scram-sha-512    simple          
# kubectl -n acl-test get secrets 
NAME                                     TYPE                                  DATA   AGE
default-token-ml5nz                      kubernetes.io/service-account-token   3      149m
my-cluster-clients-ca                    Opaque                                1      148m
my-cluster-clients-ca-cert               Opaque                                3      148m
my-cluster-cluster-ca                    Opaque                                1      148m
my-cluster-cluster-ca-cert               Opaque                                3      148m
my-cluster-cluster-operator-certs        Opaque                                4      148m
my-cluster-entity-operator-certs         Opaque                                4      146m
my-cluster-entity-operator-token-m99gm   kubernetes.io/service-account-token   3      146m
my-cluster-kafka-brokers                 Opaque                                4      147m
my-cluster-kafka-token-w2gr8             kubernetes.io/service-account-token   3      147m
my-cluster-zookeeper-nodes               Opaque                                4      148m
my-cluster-zookeeper-token-8rb4p         kubernetes.io/service-account-token   3      148m
strimzi-cluster-operator-token-gcbfz     kubernetes.io/service-account-token   3      149m

And it also seems strange to me that the Ready field of kafkauser is empty.
So the question is, why isn't the secret being generated?

I've tried creating the secret manually, this way:

apiVersion: v1
kind: Secret
metadata:
  name: user1
  labels:
    strimzi.io/kind: KafkaUser
    strimzi.io/cluster: my-cluster
type: Opaque
data:
  password: cGFzc3dvcmQK
  sasl.jaas.config: b3JnLmFwYWNoZS5rYWZrYS5jb21tb24uc2VjdXJpdHkuc2NyYW0uU2NyYW1Mb2dpbk1vZHVsZSByZXF1aXJlZCB1c2VybmFtZT11c2VyMSBwYXNzd29yZD1wYXNzd29yZDsK

And trying to connect using kafka-console-producer with the following config file:

# cat client.cfg 
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username=user1 password=password;

# ./bin/kafka-console-producer.sh --bootstrap-server 100.127.240.4:32452 --producer.config client.cfg --topic my-topic

but I get an authorization error

ERROR [Producer clientId=console-producer] Connection to node -1 (100.127.240.4/100.127.240.4:32452) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)

Please tell me what could be the error? Maybe there is a step-by-step instruction with the same setup example?

[scholzj]

The strimzi.io/cluster label on your KafkaUser resource needs to match the name of your Kafka cluster which is my-cluster and not acl-test.

[sleepdan]

# kubectl -n acl-test get secrets user1 
NAME    TYPE     DATA   AGE
user1   Opaque   2      78s
# kubectl -n acl-test create -f create_user 
kafkauser.kafka.strimzi.io/user1 created
# kubectl -n acl-test get secrets user1 
NAME    TYPE     DATA   AGE
user1   Opaque   2      97s

[scholzj]

But is the content the same as when you created it?

[sleepdan]

after create secret:

# kubectl get secrets user1 -n acl-test -o json
{
    "apiVersion": "v1",
    "data": {
        "password": "aGVsbG8K",
        "sasl.jaas.config": "b3JnLmFwYWNoZS5rYWZrYS5jb21tb24uc2VjdXJpdHkuc2NyYW0uU2NyYW1Mb2dpbk1vZHVsZSByZXF1aXJlZCB1c2VybmFtZT11c2VyMSBwYXNzd29yZD1oZWxsbzsK"
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2021-06-02T05:09:36Z",
        "labels": {
            "strimzi.io/cluster": "my-cluster",
            "strimzi.io/kind": "KafkaUser"
        },
        "name": "user1",
        "namespace": "acl-test",
        "resourceVersion": "2367044",
        "selfLink": "/api/v1/namespaces/acl-test/secrets/user1",
        "uid": "8664750a-a842-49a4-bcd7-8686345a5427"
    },
    "type": "Opaque"
}

after create user:

# kubectl get secrets user1 -n acl-test -o json
{
    "apiVersion": "v1",
    "data": {
        "password": "aGVsbG8K",
        "sasl.jaas.config": "b3JnLmFwYWNoZS5rYWZrYS5jb21tb24uc2VjdXJpdHkuc2NyYW0uU2NyYW1Mb2dpbk1vZHVsZSByZXF1aXJlZCB1c2VybmFtZT0idXNlcjEiIHBhc3N3b3JkPSJoZWxsbwoiOw=="
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2021-06-02T05:09:36Z",
        "labels": {
            "app.kubernetes.io/instance": "user1",
            "app.kubernetes.io/managed-by": "strimzi-user-operator",
            "app.kubernetes.io/name": "strimzi-user-operator",
            "app.kubernetes.io/part-of": "strimzi-user1",
            "strimzi.io/cluster": "my-cluster",
            "strimzi.io/kind": "KafkaUser"
        },
        "name": "user1",
        "namespace": "acl-test",
        "ownerReferences": [
            {
                "apiVersion": "kafka.strimzi.io/v1beta2",
                "blockOwnerDeletion": false,
                "controller": false,
                "kind": "KafkaUser",
                "name": "user1",
                "uid": "80e99af2-8970-46b3-ada7-85a0043ba71a"
            }
        ],
        "resourceVersion": "2367399",
        "selfLink": "/api/v1/namespaces/acl-test/secrets/user1",
        "uid": "8664750a-a842-49a4-bcd7-8686345a5427"
    },
    "type": "Opaque"
}

as you can see the passwords are the same, but "sasl.jaas.config" are different. If i do base64 --decode result is slightly different:

# echo "b3JnLmFwYWNoZS5rYWZrYS5jb21tb24uc2VjdXJpdHkuc2NyYW0uU2NyYW1Mb2dpbk1vZHVsZSByZXF1aXJlZCB1c2VybmFtZT11c2VyMSBwYXNzd29yZD1oZWxsbzsK" | base64 -d
org.apache.kafka.common.security.scram.ScramLoginModule required username=user1 password=hello;
# echo "b3JnLmFwYWNoZS5rYWZrYS5jb21tb24uc2VjdXJpdHkuc2NyYW0uU2NyYW1Mb2dpbk1vZHVsZSByZXF1aXJlZCB1c2VybmFtZT0idXNlcjEiIHBhc3N3b3JkPSJoZWxsbwoiOw==" | base64 -d
org.apache.kafka.common.security.scram.ScramLoginModule required username="user1" password="hello
";

i try use this string org.apache.kafka.common.security.scram.ScramLoginModule required username="user1" password="hello"; in config for kafka-console-producer but still got error:

# cat client.cfg 
group.id=my-group
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="user1" password="hello";

error: ERROR [Producer clientId=console-producer] Connection to node -1 (100.127.240.4/100.127.240.4:32452) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512 (org.apache.kafka.clients.NetworkClient)

[scholzj]

I think your password is wrong. It has a new line character at the end. You should use just aGVsbG8= instead. With your current setting, the new-line character is part of the password, but the SASL JAAS configuration probably cannot handle it.

[sleepdan]

Thank you!!! need use -n option for echo..
its work!