Transport Layer Security (TLS) is a form of public key cryptography. By default, Pulsar clients communicate with Pulsar services in plain text. This means that all data is sent in the clear. You can use TLS to encrypt this traffic to protect the traffic from the snooping of a man-in-the-middle attacker.
This section introduces how to configure TLS encryption in Pulsar. For how to configure mTLS authentication in Pulsar, refer to mTLS authentication. Alternatively, you can use another Athenz authentication on top of TLS transport encryption.
note
Enabling TLS encryption may impact the performance due to encryption overhead.
TLS certificates include the following three types. Each certificate (key pair) contains both a public key that encrypts messages and a private key that decrypts messages.
Certificate Authority (CA)
CA private key is distributed to all parties involved.
CA public key (trust cert) is used for signing a certificate for either broker or clients.
Server key pairs
Client key pairs (for mutual TLS)
For both server and client certificates, the private key with a certificate request is generated first, and the public key (the certificate) is generated after the trust cert signs the certificate request. When mTLS authentication is enabled, the server uses the trust cert to verify that the client has a key pair that the certificate authority signs. The Common Name (CN) of a client certificate is used as the client's role token, while the Subject Alternative Name (SAN) of a server certificate is used for Hostname verification.
note
The validity of these certificates is 365 days. It's highly recommended to use sha256 or sha512 as the signature algorithm, while sha1 is not supported.
Hostname verification is a TLS security feature whereby a client can refuse to connect to a server if the Subject Alternative Name (SAN) does not match the hostname that the hostname is connecting to.
By default, Pulsar clients disable hostname verification, as it requires that each broker has a DNS record and a unique cert.
One scenario where you may want to enable hostname verification is where you have multiple proxy nodes behind a VIP, and the VIP has a DNS record, for example, pulsar.mycompany.com. In this case, you can generate a TLS cert with pulsar.mycompany.com as the SAN, and then enable hostname verification on the client.
To enable hostname verification in Pulsar, ensure that SAN exactly matches the fully qualified domain name (FQDN) of the server. The client compares the SAN with the DNS domain name to ensure that it is connecting to the desired server. See Configure clients for more details.
Moreover, as the administrator has full control of the CA, a bad actor is unlikely to be able to pull off a man-in-the-middle attack. allowInsecureConnection allows the client to connect to servers whose cert has not been signed by an approved CA. The client disables allowInsecureConnection by default, and you should always disable allowInsecureConnection in production environments. As long as you disable allowInsecureConnection, a man-in-the-middle attack requires that the attacker has access to the CA.
You can use a certificate authority (CA) to sign both server and client certificates. This ensures that each party trusts the others. Store CA in a very secure location (ideally completely disconnected from networks, air-gapped, and fully encrypted).
To configure hostname verification, you need to enter the hostname of the server in alt_names as the Subject Alternative Name (SAN). To ensure that multiple machines can reuse the same certificate, you can also use a wildcard to match a group of server hostnames, for example, *.server.usw.example.com.
At this point, you have a cert, server.cert.pem, and a key, server.key-pk8.pem, which you can use along with ca.cert.pem to configure TLS encryption for your brokers and proxies.
At this point, you have a cert broker_client.cert.pem and a key broker_client.key-pk8.pem, which you can use along with ca.cert.pem to configure TLS encryption for your broker client.
At this point, you have a cert admin.cert.pem and a key admin.key-pk8.pem, which you can use along with ca.cert.pem to configure TLS encryption for your pulsar admin.
At this point, you have a cert client.cert.pem and a key client.key-pk8.pem, which you can use along with ca.cert.pem to configure TLS encryption for your client.
At this point, you have a cert proxy.cert.pem and a key proxy.key-pk8.pem, which you can use along with ca.cert.pem to configure TLS encryption for your proxy.
To configure a Pulsar broker to use TLS encryption, you need to add these values to broker.conf in the conf directory of your Pulsar installation. Substitute the appropriate certificate paths where necessary.
# configure TLS ports brokerServicePortTls=6651 webServicePortTls=8081 # configure CA certificate tlsTrustCertsFilePath=/path/to/ca.cert.pem # configure server certificate tlsCertificateFilePath=/path/to/server.cert.pem # configure server's priviate key tlsKeyFilePath=/path/to/server.key-pk8.pem # enable mTLS tlsRequireTrustedClientCertOnConnect=true # configure mTLS for the internal client brokerClientTlsEnabled=true brokerClientTrustCertsFilePath=/path/to/ca.cert.pem brokerClientCertificateFilePath=/path/to/broker_client.cert.pem brokerClientKeyFilePath=/path/to/broker_client.key-pk8.pem
To configure the broker (and proxy) to require specific TLS protocol versions and ciphers for TLS negotiation, you can use the TLS protocol versions and ciphers to stop clients from requesting downgraded TLS protocol versions or ciphers that may have weaknesses.
By default, Pulsar uses OpenSSL when it is available, otherwise, Pulsar defaults back to the JDK implementation. OpenSSL currently supports TLSv1.1, TLSv1.2 and TLSv1.3. You can acquire a list of supported ciphers from the OpenSSL ciphers command, i.e. openssl ciphers -tls1_3.
Both the TLS protocol versions and cipher properties can take multiple values, separated by commas. The possible values for protocol versions and ciphers depend on the TLS provider that you are using.
tlsProtocols=TLSv1.3,TLSv1.2: List out the TLS protocols that you are going to accept from clients. By default, it is not set.
tlsCiphers=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256: A cipher suite is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS network protocol. By default, it is null. See OpenSSL Ciphers and JDK Ciphers for more details.
For JDK 11, you can obtain a list of supported values from the documentation:
To enable TLS encryption, you need to configure the clients to use https:// with port 8443 for the web service URL, and pulsar+ssl:// with port 6651 for the broker service URL.
As the server certificate that you generated above does not belong to any of the default trust chains, you also need to either specify the path of the trust cert (recommended) or enable the clients to allow untrusted server certs.
The following examples show how to configure TLS encryption for Java/Python/C++/Node.js/C#/WebSocket clients.
Java
Python
C++
Node.js
C#
WebSocket API
importorg.apache.pulsar.client.api.PulsarClient; PulsarClient client =PulsarClient.builder() .serviceUrl("pulsar+ssl://broker.example.com:6651/") .tlsKeyFilePath("/path/to/client.key-pk8.pem") .tlsCertificateFilePath("/path/to/client.cert.pem") .tlsTrustCertsFilePath("/path/to/ca.cert.pem") .enableTlsHostnameVerification(false)// false by default, in any case .allowTlsInsecureConnection(false)// false by default, in any case .build();
from pulsar import Client client = Client("pulsar+ssl://broker.example.com:6651/", tls_hostname_verification=False, tls_trust_certs_file_path="/path/to/ca.cert.pem", tls_allow_insecure_connection=False)// defaults to false from v2.2.0 onwards
var certificate =newX509Certificate2("ca.cert.pem"); var client = PulsarClient.Builder() .TrustedCertificateAuthority(certificate)//If the CA is not trusted on the host, you can add it explicitly. .VerifyCertificateAuthority(true)//Default is 'true' .VerifyCertificateName(false)//Default is 'false' .Build();
note
VerifyCertificateName refers to the configuration of hostname verification in the C# client.
In addition to the required configurations in the conf/client.conf file, you need to configure more parameters in the conf/broker.conf file to enable TLS encryption on WebSocket service. For more details, see security settings for WebSocket.
To configure hostname verification, you need to append -ext SAN=IP:127.0.0.1,IP:192.168.20.2,DNS:broker.example.com to the value of BROKER_COMMON_PARAMS as the Subject Alternative Name (SAN).
Configure the following parameters in the conf/broker.conf file and restrict access to the store files via filesystem permissions.
brokerServicePortTls=6651 webServicePortTls=8081 # Trusted client certificates are required to connect TLS # Reject the Connection if the Client Certificate is not trusted. # In effect, this requires that all connecting clients perform TLS client # authentication. tlsRequireTrustedClientCertOnConnect=true tlsEnabledWithKeyStore=true # key store tlsKeyStoreType=JKS tlsKeyStore=/var/private/tls/broker.keystore.jks tlsKeyStorePassword=brokerpw # trust store tlsTrustStoreType=JKS tlsTrustStore=/var/private/tls/broker.truststore.jks tlsTrustStorePassword=brokerpw # internal client/admin-client config brokerClientTlsEnabled=true brokerClientTlsEnabledWithKeyStore=true brokerClientTlsTrustStoreType=JKS brokerClientTlsTrustStore=/var/private/tls/client.truststore.jks brokerClientTlsTrustStorePassword=clientpw brokerClientTlsKeyStoreType=JKS brokerClientTlsKeyStore=/var/private/tls/client.keystore.jks brokerClientTlsKeyStorePassword=clientpw
To disable non-TLS ports, you need to set the values of brokerServicePort and webServicePort to empty.
note
The default value of tlsRequireTrustedClientCertOnConnect is false, which represents one-way TLS. When it's set to true (mutual TLS is enabled), brokers/proxies require trusted client certificates; otherwise, brokers/proxies reject connection requests from clients.
importorg.apache.pulsar.client.api.PulsarClient; PulsarClient client =PulsarClient.builder() .serviceUrl("pulsar+ssl://broker.example.com:6651/") .useKeyStoreTls(true) .tlsTrustStoreType("JKS") .tlsTrustStorePath("/var/private/tls/client.truststore.jks") .tlsTrustStorePassword("clientpw") .tlsKeyStoreType("JKS") .tlsKeyStorePath("/var/private/tls/client.keystore.jks") .tlsKeyStorePassword("clientpw") .enableTlsHostnameVerification(false)// false by default, in any case .allowTlsInsecureConnection(false)// false by default, in any case .build();
note
If you set useKeyStoreTls to true, be sure to configure tlsTrustStorePath.
PulsarAdmin amdin =PulsarAdmin.builder().serviceHttpUrl("https://broker.example.com:8443") .tlsTrustStoreType("JKS") .tlsTrustStorePath("/var/private/tls/client.truststore.jks") .tlsTrustStorePassword("clientpw") .tlsKeyStoreType("JKS") .tlsKeyStorePath("/var/private/tls/client.keystore.jks") .tlsKeyStorePassword("clientpw") .enableTlsHostnameVerification(false)// false by default, in any case .allowTlsInsecureConnection(false)// false by default, in any case .build();