Neo4j – Kafka – JAAS

A couple of weeks ago, I was asked how to configure Neo4j to use JAAS with Kafka using SASL_Plaintext. While the Neo4j documentation does talk about SSL configuration, it doesn’t specifically discuss JAAS.

On the Kafka side, I used a Bitnami AMI (Kafka – AMI ID bitnami-kafka-2.3.0-0-linux-debian-9-x86_64-hvm-ebs-nami (ami-0ca61ab6a3b990db7)) running on AWS. There were some configuration changes I needed to make to enable my local Neo4j instance to connect.

Edit producer.properties and set the bootstrap.servers property to the public ip address.

bootstrap.servers=18.188.84.xxx:9092

On the server.properties file, I edited it as follows:

############################# Socket Server Settings #############################

# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
listeners=EXTERNAL://0.0.0.0:9092,INTERNAL://0.0.0.0:9093,CLIENT://0.0.0.0:9094
listener.security.protocol.map=EXTERNAL:SASL_PLAINTEXT,INTERNAL:PLAINTEXT,CLIENT:SASL_PLAINTEXT

#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092

# Hostname and port the broker will advertise to producers and consumers. If not set,
# it uses the value for "listeners" if configured.  Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
advertised.listeners=EXTERNAL://18.188.84.xxx:9092,INTERNAL://172.31.43.xxx:9093,CLIENT://18.188.84.xxx:9094

zookeeper.connect=18.188.84.xxx:2181

sasl.mechanism.inter.broker.protocol=PLAIN

sasl.enabled.mechanisms=PLAIN
#security.inter.broker.protocol=SASL_PLAINTEXT
inter.broker.listener.name=INTERNAL

On the Neo4j side, I copied the contents of /home/bitnami/stack/kafka/conf/kafka_jaas.conf and saved it to a file called kafka_client_jaas.conf in the /conf directory on my Neo4j server.

In the neo4j.conf file, I edited it as follows:

dbms.security.procedures.unrestricted=apoc.*, streams.*
dbms.security.procedures.whitelist=apoc.*, streams.*

kafka.bootstrap.servers=18.188.84.xxx:9092
kafka.zookeeper.connect=18.188.84.xxx:2181

kafka.security.protocol=SASL_PLAINTEXT
kafka.sasl.mechanism=PLAIN

streams.source.enabled.from.neo4j=true
streams.procedures.enabled.from.neo4j=true

dbms.jvm.additional=-Djava.security.auth.login.config=/Users/davidfauth/neo4j-enterprise-4.0.4_kafka/conf/kafka_client_jaas.conf

After restarting Neo4j, I was able to write to the topic on my Kafka cluster by running:

CALL streams.publish('numtest', 'Hello World2 from Neo4j!')

Good luck using Neo4j and Kafka. Drop me a comment if you have any questions.