NAIS Topic reference¶

This document describes all possible configuration values in the Topic spec, commonly known as the topic.yaml file.

acl¶

Type: array
Required: true

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

acl[].access¶

Access type granted for a application. Defaults to readwrite.

Type: enum
Required: true
Allowed values: read, readwrite, write

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

acl[].application¶

The name of the specified application

Type: string
Required: true

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

acl[].team¶

The team of the specified application

Type: string
Required: true

spec:
  acl:
    - access: read
      application: consumer
      team: otherteam
    - access: write
      application: producer
      team: myteam
    - access: readwrite
      application: processor
      team: myteam

config¶

Type: object
Required: false

spec:
  config:
    cleanupPolicy: delete
    localRetentionBytes: 1000
    localRetentionHours: 68
    maxCompactionLagMs: 60000
    maxMessageBytes: 1.048588e+06
    minCompactionLagMs: 10000
    minimumInSyncReplicas: 2
    partitions: 1
    replication: 3
    retentionBytes: 6000
    retentionHours: 168
    segmentHours: 168

config.cleanupPolicy¶

CleanupPolicy is either "delete" or "compact" or both. This designates the retention policy to use on old log segments.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_cleanup.policy

Type: enum
Required: false
Default value: delete
Allowed values: compact, compact,delete, delete

spec:
  config:
    cleanupPolicy: delete

config.localRetentionBytes¶

When set, remote storage will be used to store log segments. This value controls the size of the log that is kept before it is moved to remote storage. Must be less than RetentionBytes Not supported when CleanupPolicy is set to "compact"

Relevant information:

• https://kafka.apache.org/documentation.html#topicconfigs_local.retention.bytes

Type: integer
Required: false
Default value: -2

spec:
  config:
    localRetentionBytes: 1000

config.localRetentionHours¶

When set, remote storage will be used to store log segments. This value controls the number of hours to keep before it is moved to remote storage. Must be less than RetentionHours. Not supported when CleanupPolicy is set to "compact"

Relevant information:

• https://kafka.apache.org/documentation.html#topicconfigs_local.retention.ms

Type: integer
Required: false
Default value: -2
Maximum value: 2147483648

spec:
  config:
    localRetentionHours: 68

config.maxCompactionLagMs¶

MaxCompactionLagMs indicates the maximum time a message will remain ineligible for compaction in the log

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_max.compaction.lag.ms

Type: integer
Required: false
Default value: Inf
Minimum value: 0

spec:
  config:
    maxCompactionLagMs: 60000

config.maxMessageBytes¶

The largest record batch size allowed by Kafka (after compression if compression is enabled). If this is increased and there are consumers older than 0.10.2, the consumers' fetch size must also be increased so that they can fetch record batches this large. In the latest message format version, records are always grouped into batches for efficiency. In previous message format versions, uncompressed records are not grouped into batches and this limit only applies to a single record in that case.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_max.message.bytes

Type: integer
Required: false
Default value: 1048588
Value range: 1-5242880

spec:
  config:
    maxMessageBytes: 1.048588e+06

config.minCleanableDirtyRatioPercent¶

MinCleanableDirtyRatio indicates the minimum ratio of dirty log to retention size to initiate log compaction

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_min.cleanable.dirty.ratio

Type: integer or string
Required: false
Default value: 50%

config.minCompactionLagMs¶

MinCompactionLagMs indicates the minimum time a message will remain uncompacted in the log

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_min.compaction.lag.ms

Type: integer
Required: false
Default value: 0
Minimum value: 0

spec:
  config:
    minCompactionLagMs: 10000

config.minimumInSyncReplicas¶

When a producer sets acks to "all" (or "-1"), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_min.insync.replicas

Type: integer
Required: false
Default value: 2
Value range: 1-7

spec:
  config:
    minimumInSyncReplicas: 2

config.partitions¶

The default number of log partitions per topic.

Type: integer
Required: false
Default value: 1
Value range: 1-1000000

spec:
  config:
    partitions: 1

config.replication¶

The default replication factor for created topics.

Relevant information:

• https://kafka.apache.org/33/documentation.html#replication

Type: integer
Required: false
Default value: 3
Minimum value: 2

spec:
  config:
    replication: 3

config.retentionBytes¶

Configuration controls the maximum size a partition can grow to before we will discard old log segments to free up space if we are using the "delete" retention policy. By default there is no size limit only a time limit. Since this limit is enforced at the partition level, multiply it by the number of partitions to compute the topic retention in bytes.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_retention.bytes

Type: integer
Required: false
Default value: -1

spec:
  config:
    retentionBytes: 6000

config.retentionHours¶

The number of hours to keep a log file before deleting it.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_retention.ms

Type: integer
Required: false
Default value: 168
Maximum value: 2147483648

spec:
  config:
    retentionHours: 168

config.segmentHours¶

The number of hours after which Kafka will force the log to roll even if the segment file isn't full to ensure that retention can delete or compact old data.

Relevant information:

• https://kafka.apache.org/33/documentation.html#topicconfigs_segment.ms

Type: integer
Required: false
Default value: 168
Value range: 1-8760

spec:
  config:
    segmentHours: 168

pool¶

Type: string
Required: true

spec:
  pool: dev-nais-dev