How to use secret provider with SASL_SSL/JAAS authentication?

[jslusher]

On my KafkaConnect and KafkaConnector I have the following set up:

    config.providers: secrets,configmaps
    config.providers.secrets.class: io.strimzi.kafka.KubernetesSecretConfigProvider
    config.providers.configmaps.class: io.strimzi.kafka.KubernetesConfigMapConfigProvider

The rolebindings are in place and I don't seem to be getting any errors related to permissions to the secret from the service account.

The secret I'm referring to looks like this:

sasl-auth: sasl-auth=org.apache.kafka.common.security.scram.ScramLoginModule required username="kc-test" password="xxxxxxxxxxxxx";

I have a Debezium connector I'm trying to configure using SSL_SASL with a JAAS config using this secret:

    database.history.producer.sasl.jaas.config: ${secrets:strimzi-connect-secrets:sasl-auth}
    database.history.consumer.sasl.jaas.config: ${secrets:strimzi-connect-secrets:sasl-auth}

Looking at the config using the REST api it looks like it might be taking the setting literally?

kx ksqldb-test-connect-fdfd8fd6-5ht2z -- curl http://localhost:8083/connectors/debezium.discogs/config | jq
{
...
  "database.history.consumer.sasl.jaas.config": "${secrets:strimzi-connect-secrets:sasl-auth}",
 ...
  "database.history.producer.sasl.jaas.config": "${secrets:strimzi-connect-secrets:sasl-auth}",

I get the following error when the connector starts up:

...
Caused by: java.lang.IllegalArgumentException: Login module control flag is not available in the JAAS config
...

Which I think means the settings are malformed. Am I doing this right?

[scholzj]

The REST API will not show you the actual values. It would show you the provider. I think this is expected (as you do not really want to see the secret). TBH, without full logs etc., it is hard to say if the error comes because the provider value is not replaced or whether it just doesn't like your value from the secret. Best way to try it might be to use it on some field which is not secret and will be printed on the Kafka log. There you can see what value is actually set there. With JAAS config, it will be masked so you won't see anything in the log.

[jslusher]

I added a parameter called misc.config.setting and I can see that the value of the secret did render in the logs:

stern ksqldb-test-connect | grep misc
...
ksqldb-test-connect-fdfd8fd6-txb9t ksqldb-test-connect 2023-04-04 23:49:45,460 INFO [debezium.company|task-0]    misc.config.setting = sasl-auth=org.apache.kafka.common.security.scram.ScramLoginModule required username="kc-test" password="xxxxxxxxxxx"; (io.debezium.connector.common.BaseSourceTask) [task-thread-debezium.company-0]

The value it has is what I expect, at least up to the semicolon. I'm not sure what that is past the semicolon ( (io.debezium.connector.common.BaseSourceTask) [task-thread-debezium.company-0]). Is that to be expected? Looks like that suffixes the other config entries in the kafka logs, so I'm guessing that part is normal.

[scholzj]

TBH, I'm not sure I never used SASL this way. Escaping was always problem with it. It is also sensitive for special characters in the password. But I'm afraid I do not have a simple answer why does it not like it :-(.

I'm not sure how did you get the log and how the rest of it looks like -> so I'm not sure about the part after the semicolon. But it looks like the log output is mixed and this comes from a different log messages.

You had to base64 encode the the JAAs string for the secret. Make sure you do not have some newline character, carriage return, or something like that at the end of the line. That might cause the error and the log file and it is a common problem when base64 encoding the strings - I saw it before with passwords.

[jslusher]

I got this working with the following settings:

    database.history.producer.security.protocol: SASL_SSL
    database.history.producer.sasl.mechanism: SCRAM-SHA-512
    database.history.producer.sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="kc-test" password="${secrets:strimzi-connect-secrets:kc-test-password}";
    database.history.consumer.security.protocol: SASL_SSL
    database.history.consumer.sasl.mechanism: SCRAM-SHA-512
    database.history.consumer.sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="kc-test" password="${secrets:strimzi-connect-secrets:kc-test-password}";

The original problem I had was that I was copying the sasl config settings from a fileConfigProvider configuration, which requires the additional sasl-auth= in front of the actual setting. You'll notice it in there in the paste above from the connect logs. Thanks for the tip about the fake setting to see that actual value, that helped me finally determine that it was the string itself that was the problem.