NAIS Topic reference

This document describes all possible configuration values in the Topic spec, commonly known as the topic.yaml file.

acl

Type: array
Required: true

Example

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

acl[].access

Access type granted for a application. Defaults to readwrite.

Type: enum
Required: true
Allowed values: read, readwrite, write

Example

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

acl[].application

The name of the specified application

Type: string
Required: true

Example

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

acl[].team

The team of the specified application

Type: string
Required: true

Example

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

config

Type: object
Required: false

Example

spec:
  config:
    cleanupPolicy: delete
    localRetentionBytes: 1000
    localRetentionHours: 68
    maxCompactionLagMs: 60000
    maxMessageBytes: 1.048588e+06
    minCompactionLagMs: 10000
    minimumInSyncReplicas: 2
    partitions: 1
    replication: 3
    retentionBytes: 6000
    retentionHours: 168
    segmentHours: 168

config.cleanupPolicy

CleanupPolicy is either "delete" or "compact" or both. This designates the retention policy to use on old log segments.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_cleanup.policy

Type: enum
Required: false
Default value: delete
Allowed values: compact, compact,delete, delete

Example

spec:
  config:
    cleanupPolicy: delete

config.localRetentionBytes

When set, remote storage will be used to store log segments. This value controls the size of the log that is kept before it is moved to remote storage. Must be less than RetentionBytes

Relevant information:

• https://kafka.apache.org/documentation.html#topicconfigs_local.retention.bytes

Type: integer
Required: false
Default value: -2

Example

spec:
  config:
    localRetentionBytes: 1000

config.localRetentionHours

When set, remote storage will be used to store log segments. This value controls the number of hours to keep before it is moved to remote storage. Must be less than RetentionHours.

Relevant information:

• https://kafka.apache.org/documentation.html#topicconfigs_local.retention.ms

Type: integer
Required: false
Default value: -2
Maximum value: 2147483648

Example

spec:
  config:
    localRetentionHours: 68

config.maxCompactionLagMs

MaxCompactionLagMs indicates the maximum time a message will remain ineligible for compaction in the log

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_max.compaction.lag.ms

Type: integer
Required: false
Default value: Inf
Minimum value: 0

Example

spec:
  config:
    maxCompactionLagMs: 60000

config.maxMessageBytes

The largest record batch size allowed by Kafka (after compression if compression is enabled). If this is increased and there are consumers older than 0.10.2, the consumers' fetch size must also be increased so that they can fetch record batches this large. In the latest message format version, records are always grouped into batches for efficiency. In previous message format versions, uncompressed records are not grouped into batches and this limit only applies to a single record in that case.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_max.message.bytes

Type: integer
Required: false
Default value: 1048588
Value range: 1-5242880

Example

spec:
  config:
    maxMessageBytes: 1.048588e+06

config.minCleanableDirtyRatioPercent

MinCleanableDirtyRatio indicates the minimum ratio of dirty log to retention size to initiate log compaction

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_min.cleanable.dirty.ratio

Type: integer or string
Required: false
Default value: 50%

config.minCompactionLagMs

MinCompactionLagMs indicates the minimum time a message will remain uncompacted in the log

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_min.compaction.lag.ms

Type: integer
Required: false
Default value: 0
Minimum value: 0

Example

spec:
  config:
    minCompactionLagMs: 10000

config.minimumInSyncReplicas

When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_min.insync.replicas

Type: integer
Required: false
Default value: 2
Value range: 1-7

Example

spec:
  config:
    minimumInSyncReplicas: 2

config.partitions

The default number of log partitions per topic.

Type: integer
Required: false
Default value: 1
Value range: 1-1000000

Example

spec:
  config:
    partitions: 1

config.replication

The default replication factor for created topics.

Relevant information:

• https://kafka.apache.org/33/documentation.html#replication

Type: integer
Required: false
Default value: 3
Minimum value: 2

Example

spec:
  config:
    replication: 3

config.retentionBytes

Configuration controls the maximum size a partition can grow to before we will discard old log segments to free up space if we are using the "delete" retention policy. By default there is no size limit only a time limit. Since this limit is enforced at the partition level, multiply it by the number of partitions to compute the topic retention in bytes.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_retention.bytes

Type: integer
Required: false
Default value: -1

Example

spec:
  config:
    retentionBytes: 6000

config.retentionHours

The number of hours to keep a log file before deleting it.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_retention.ms

Type: integer
Required: false
Default value: 168
Maximum value: 2147483648

Example

spec:
  config:
    retentionHours: 168

config.segmentHours

The number of hours after which Kafka will force the log to roll even if the segment file isn't full to ensure that retention can delete or compact old data.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_segment.ms

Type: integer
Required: false
Default value: 168
Value range: 1-8760

Example

spec:
  config:
    segmentHours: 168

pool

Type: string
Required: true

Example

spec:
  pool: dev-nais-dev