Skip to content

Kafka logging for Gluu#


This tutorial offers a step-by-step guide for using Apache Kafka to aggregate logs for oxAuth and oxTrust.


  1. From inside the chroot, unzip the oxAuth and Identity war files.

    # unzip -p /opt/gluu/jetty/oxauth/webapps/oxauth.war WEB-INF/classes/log4j2.xml > /opt/gluu/jetty/oxauth/resources/log4j2.xml
    # unzip -p /opt/gluu/jetty/identity/webapps/identity.war WEB-INF/classes/log4j2.xml > /opt/gluu/jetty/identity/resources/log4j2.xml
  2. Download the Kafka Clients jar file and copy it into oxAuth and Identity.

    # mkdir /tmp/jars
    # wget -O /tmp/jars/kafka-clients-2.2.0.jar
    # wget -O /tmp/jars/lz4-java-1.5.0.jar
    # wget -O /tmp/jars/slf4j-api-1.7.25.jar
    # wget -O /tmp/jars/snappy-java-
    # wget -O /tmp/jars/zstd-jni-1.3.8-1.jar
    # cp /tmp/jars/*.jar /opt/gluu/jetty/oxauth/lib/ext/
    # cp /tmp/jars/*.jar /opt/gluu/jetty/identity/lib/ext/
    # rm -rf /tmp/jars
  3. Make the following changes to log4j2.xml in both /opt/gluu/jetty/oxauth/resources and /opt/gluu/jetty/identity/resources:

    • Under section appenders, add:

      <Kafka name="Kafka" topic="my-gluu-topic">
      <PatternLayout pattern="%d %-5p [oxauth] [%t] [%C{6}] (%F:%L) - %m%n"/>  
      <Property name="bootstrap.servers"=>KAFKA-1.MY.DOMAIN:9094,KAFKA-2.MY.DOMAIN:9094,...</Property>
      <Property name="compression.type">gzip</Property>
      <Property name="ssl.truststore.location">/PATH/TO/truststore.jks</Property>
      <Property name="ssl.truststore.password">PASSWORD_FOR_TRUSTSTORE</Property>
      <Property name="security.protocol">SSL</Property>
    • Under each logger, including the root logger, in sections that have an AppenderRef, add:

      <AppenderRef ref="Kafka" />
  4. Add "-dlog4j.configurationfile=resources/log4j2.xml" to java_options in both /etc/default/oxauth and /etc/default/identity

  5. Restart systemd, oxAuth, and Identity

    # service oxauth restart && service identity restart
    # logout
    # systemctl daemon-reload