Skip to content

Kafka

Warning

This feature applies only to Aiven hosted Kafka. On-premises Kafka is deprecated, and creating new topics on-premises was disabled summer 2021. For on-premises Kafka, see on-premises Kafka documentation.

Warning

For the time being, Kafka Streams is not supported on Aiven. This is due to how Kafka Streams creates temporary topics. We have a plan to solve this, but it is not implemented yet.

Abstract

NAV uses Aiven hosted Kafka. Access to Kafka is granted by defining a Topic resource in one of our Kubernetes clusters.

Upon defining a Topic, Kafkarator will create the Topic in one of the Kafka pools. A pool is a highly available, replicated Kafka cluster running at Aiven. After the topic is created, Kafkarator will add relevant users to the topic's access control list (ACL).

When an application that uses Kafka is deployed, Aivenator will orchestrate generation of users and credentials. These credentials are made available to applications through a Secret in the relevant team namespace. This secret is automatically mounted by Naiserator into application pods as environment variables and files.

For a list of variables, see accessing topics from an application.

Status and roadmap

  • Release status: Generally Available
  • Availability: NAIS GCP, NAIS on-premises, legacy infrastructure on-premises

Follow development on the PIG-Aiven Trello board.