A Complete Guide to Securely Connecting Kafka and Kafka Command Line Tools Using Mutual TLS
How to use TLS, client authentication, and CA certificates in Kafka and Kafka Command Line Tools
Create a private key and request a certificate for your Kafka server
Before you can teach your server to speak TLS, you will need a certificate issued by a trusted certificate authority (CA). If your organization already runs its own CA and you have a private key and certificate for your Kafka server, along with your CA's root certificate, you can skip to the next step.
To request a certificate from your CA using the step CLI, bootstrap your CA with step ca bootstrap and run the following command (sub the server name for the actual name / DNS name of your Kafka server).
$ step ca certificate "myserver.internal.net" server.crt server.keyYour certificate and private key will be saved in server.crt and server.key respectively.
Request a copy of your CA root certificate, which will be used to make sure each application can trust certificates presented by other applications.
$ step ca root ca.crtYour certificate will be saved in ca.crt.
Configure Kafka to authenticate itself with its TLS certificate
We now want to instruct our Kafka server to identify itself using the certificate issued in the last step and to force clients to connect over TLS.
We'll demonstrate how to configure an individual Kafka broker/machine. You'll need to issue a new certificate and repeat these steps for each broker in your Kafka cluster.
Use openssl to package up the server private key and certificate into PKCS12 format. You'll be prompted to create a password here. Hold on to this, as you'll need it in the next step and in configuration later.
$ openssl pkcs12 -export -in server.crt -inkey server.key -name myserver.internal.net > server.p12Next, use keytool to create a Java KeyStore (JKS) with the certificate and key for use by Kafka. You'll be prompted to create a new password for the resulting file as well as enter the password for the PKCS12 file from the previous step. Hang onto the new JKS password for use in configuration below.
$ keytool -importkeystore -srckeystore server.p12 -destkeystore kafka.server.keystore.jks -srcstoretype pkcs12 -alias myserver.internal.netNote: It's safe to ignore the following warning from keytool.
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server.p12 -destkeystore kafka.server.keystore.jks -srcstoretype pkcs12".You'll also need a trust store in JKS format containing the root certificate from your CA. Kafka brokers will use this trust store to make sure certificates presented by clients and other brokers were signed by your CA. Create the password and agree to trust your CA certificate (type "yes"). Hold onto thie password for this one as well.
$ keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca.crtIn your Kafka configuration directory, modify server.properties to remove any plain text listeners and require SSL (TLS). You'll also want to require that Kafka brokers only speak to each other over TLS.
#...
listeners=SSL://:9093
security.inter.broker.protocol=SSL
#...If advertised.listeners is also configured, you may choose to either comment it out or configure it to use SSL (TLS) as well.
#...
advertised.listeners=SSL://:9093
#...Copy your Java KeyStore files into place.
$ sudo mkdir -p /var/private/ssl
$ sudo cp kafka.server.*.jks /var/private/ssl/
$ sudo chown kafka:kafka /var/private/ssl/kafka.server.*.jksReference them in server.properties and provide the passwords you created for each.
#...
ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password=<keystore password>
ssl.key.password=<pkcs12 password>
ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password=<truststore password>
#...Restart your Kafka server for your changes to take effect.
Configure Kafka to require clients to authenticate with a certificate issued by your CA
To tell Kafka to use mutual TLS and not just one-way TLS, we must instruct it to require client authentication to ensure clients present a certificate from our CA when they connect.
We'll demonstrate how to configure an individual Kafka broker/machine. You'll need to issue a new certificate and repeat these steps for each broker in your Kafka cluster.
If you haven't already, create a JKS trust store for your Kafka broker containing your root CA certificate. Kafka will use this certificate to verify any client certificates are valid and issued by your CA. Hang onto the password you create for your server configuration.
$ keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca.crtAdd configurations for the trust store to server.properties.
# ...
ssl.truststore.location=/path/to/kafka.server.truststore.jks
ssl.truststore.password=<truststore password>
# ...Lastly, configure server.properties to require that clients authenticate with certificates whenever they connect.
# ...
ssl.client.auth=required
# ...Restart your Kafka server (and possibly ZooKeeper) for your changes to take effect.
That's it! Kafka should now be able to receive TLS connections from clients who authenticate themselves using a certificate issued by your trusted CA.
Create a private key and request a certificate for your Kafka Command Line Tools client
Request a new certificate from your CA to represent your Kafka Command Line Tools client.
$ step ca certificate "myuser" client.crt client.keyYour certificate and private key will be saved in client.crt and client.key respectively.
Open a connection from Kafka Command Line Tools using mutual TLS
Now, we need only to configure our Kafka Command Line Tools client to make authenticated requests using our certificate and private key. The CA root certificate will be used to verify that the client can trust the certificate presented by the server.
Use openssl to package up the client private key and certificate into PKCS12 format. You'll be prompted to create a password here. Hold on to this, as you'll need it in the next step and in configuration later.
$ openssl pkcs12 -export -in client.crt -inkey client.key -name myuser > client.p12Next, use keytool to create a Java KeyStore (JKS) with the certificate and key. You'll be prompted to create a new password for the resulting file as well as enter the password for the PKCS12 file from the previous step. Hang onto the new JKS password for use in configuration below.
$ keytool -importkeystore -srckeystore client.p12 -destkeystore kafka.client.keystore.jks -srcstoretype pkcs12 -alias myuserNote: It's safe to ignore the following warning from keytool.
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client.p12 -destkeystore kafka.client.keystore.jks -srcstoretype pkcs12".You'll also need a trust store in JKS format containing the root certificate from your CA. The command line tools will use this trust store to make sure the certificate presented by the broker was signed by your CA. Create the password and agree to trust your CA certificate (type "yes"). Hold onto thie password for this one as well.
$ keytool -keystore kafka.client.truststore.jks -alias CARoot -import -file ca.crtCopy your Java KeyStore files into place.
$ sudo mkdir -p /var/private/ssl
$ sudo cp kafka.client.*.jks /var/private/ssl/
$ sudo chown kafka:kafka /var/private/ssl/kafka.client.*.jksCreate a new properties file named client.properties that you'll reference when you use the command line tools to open a connection. Configure it to use your key store and trust store JKS files.
security.protocol=SSL
ssl.keystore.location=/var/private/ssl/kafka.client.keystore.jks
ssl.keystore.password=<keystore password>
ssl.key.password=<pkcs12 password>
ssl.truststore.location=/var/private/ssl/kafka.client.truststore.jks
ssl.truststore.password=<truststore password>Since this file contains passwords for your key stores, we'll want to tighten up filesystem permissions.
$ chmod 0600 client.propertiesFinally, use your client.properties file when making connections to a broker from the Kafka command line tools.
$ kafka-console-consumer --bootstrap-server myserver.internal.net:9093 -topic mytopic --consumer.config client.properties
$ kafka-console-producer --broker-list myserver.internal.net:9093 -topic mytopic --producer.config client.propertiesAutomate certificate renewal
By default, step-ca issues certificates with a 24 hour expiration. Short-lived certificates have many benefits but also require that you renew your certificates each day before they expire. How you renew certificates is often dependent on how you deploy your application. See the step-ca certificate lifecycle management docs for more information.
All documentation content from the Hello mTLS project is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0).
Subscribe to updates
Unsubscribe anytime. See our privacy policy.
© 2024 Smallstep Labs, Inc. All rights reserved.