Glossary¶
Look up the terms these docs use for Narad's parts and behaviour, each with the page that covers it in full.
Acked-ahead set¶
The acks a partition holds for messages above its oldest unacked message, until the gap below them closes. It is capped per partition by max_acked_ahead_per_partition and kept on disk.
See: Consume path.
At-least-once delivery¶
Narad's delivery guarantee: an accepted message is delivered until a consumer acks it, as long as retention keeps it, and it can be delivered more than once.
See: Delivery contract.
Attach epoch¶
An ID a child topic gets each time it is attached to a parent. Fan-out progress saved under an earlier attachment is never reused, so a detach followed by an attach starts afresh.
See: Fan-out engine.
Attach point¶
The parent's position, one offset per parent partition, recorded when a child is attached. The child receives the messages from there on, never older ones.
See: Fan-out engine.
Committed frontier¶
The offset below which every message of a partition is acked. It only moves forward, and it is kept on disk.
See: Consume path.
Consume¶
A request that takes the next available message of a topic, together with a lease on it.
See: Consume and acknowledge messages and Consume messages.
Decommission¶
Draining a node before it is removed: its partitions move to the other nodes, then it leaves the Raft voters.
See: Scale out and in and Rebalance and decommission.
Delay child¶
A fan-out child that receives each parent message a fixed time after the parent committed it. Nothing can be produced to it directly.
See: Fan out and delay messages.
Delivery token¶
A marker a node leaves with each node that owns a partition of a topic, while one of its consumers waits for a message. The owner uses it to tell that node when a message arrives, so nobody polls.
See: Consume path.
Dispatch¶
The background step that moves an accepted message from the ingress WAL of the node that accepted it to the owner of its partition.
See: Produce path.
Extend¶
An ack with extend=true. It renews a lease for a full visibility timeout, for a handler that needs more time.
See: Consume and acknowledge messages.
Fan-out child¶
A topic linked to a parent topic that receives a copy of every message produced to the parent from the attach point on. It has its own partitions, consumers and retention.
See: Fan out and delay messages.
Force-promote¶
What ends a partition move whose source node stays dead: after 2 minutes, the destination takes over the partition with the copy it already holds, if that copy holds everything the source had made visible.
See: Rebalance and decommission.
Gateway node¶
The node a client's request reaches. It serves the request and forwards work to the owners of the partitions involved. For a topic it owns no partitions of, a node is only a gateway.
See: Consume path.
Grant¶
A permission: one action (produce, consume, create or admin) on the topics that match a list of patterns.
See: Access model and grants.
Hidden tail¶
Records on a partition's disk above its high watermark, which consumers cannot see, such as those written by a commit that a crash interrupted.
See: Storage engine.
High watermark¶
One past the last offset of a partition that consumers can see. Everything below it is committed and synced to disk.
See: Storage engine.
Idempotent handler¶
A message handler that has the same effect whether it runs once or several times on the same message, for example because it records an ID from the payload and skips IDs it has seen. At-least-once delivery requires one.
See: Handle retries and dead letters.
Incarnation¶
One life of a topic name, identified by the topic's id. A topic deleted and created again under the same name is a new incarnation with a new id, so nothing of the old one can mix with it.
See: Cluster lifecycle.
Ingress WAL¶
Each node's write-ahead log of the produces it accepted. A produce is written there and synced to disk before its 202, then dispatched to its partition's owner.
See: Produce path.
Key¶
An optional string sent with a produce. Messages with the same key go to the same partition while the partition count and the partition owners stay the same. It is not an ordering guarantee.
See: Produce messages.
Lane¶
New in v3.2.0. One of up to 8 ordered streams a remote child sends each parent partition on. A key always travels on one lane, and a lane sends one request at a time.
See: Remote replication.
Lease¶
The exclusive, time-limited hold a consumer gets on a message it consumed. It lasts the topic's visibility timeout, and an ack, extend or nack settles it.
See: Core concepts.
Metastore¶
The cluster's metadata (topics, users and grants, partition assignments, members), replicated to every node through Raft.
See: Metastore and Raft.
Nack¶
An ack with extend=0. It ends a lease at once, so the message can be delivered again right away.
See: Consume and acknowledge messages.
Owner¶
Two meanings, told apart by context. A partition's owner is the node that stores the partition and serves its consumers. A topic's owner is the user that created it, who may change and delete it.
See: Architecture overview and Access model and grants.
Partition¶
A slice of a topic, stored on one node. Partitions spread storage and consumption across the cluster.
See: Core concepts.
Quarantine¶
Setting a directory aside by renaming it instead of serving or deleting it. Two kinds:
- A topic directory that belongs to a deleted incarnation of a topic is renamed
<name>.stale-<id>. It is removed later, once the leader confirms that incarnation is gone. - A moved partition's old copy that the stale-copy sweep cannot prove is covered by the new owner (it is ahead of the position the partition was promoted at, or, from v3.1.0, the new owner cannot vouch for it), or, from v3.1.0, that a move back onto the node finds at the partition's path holding unexpired records, is renamed
<partition>.quarantine. Narad never serves it and removes it only when the topic is deleted; its records may exist only there, so an operator decides what to do with it.
See: Cluster lifecycle, Rebalance and decommission and Troubleshooting.
Receipt handle¶
The string a consume returns with a message, in the form partition:offset:nonce. It names the lease; send it back to ack, extend or nack the message.
See: Ack, extend or nack a message.
Remote¶
New in v3.2.0. Another Narad cluster this cluster may send to: a name, an https URL, a username and a write-only password, an optional CA bundle and limits, managed by admins through the API.
See: Manage remotes.
Remote child¶
New in v3.2.0. A fan-out child whose topic lives on a remote. On this cluster it is a stub with no partitions; the owners of the parent's partitions send every record to the remote's topic through its batch produce, at least once.
See: Replicate a topic to another cluster.
Replica child¶
A fan-out child created with parent, so that its partitions sit on other nodes than the parent's: an asynchronous second copy of a topic.
See: Back up and replicate topics.
Reroute¶
Sending accepted messages to a live sibling partition when their partition's owner is dead or keeps failing, so produce stays available. It is one of the reasons Narad does not guarantee order.
See: Produce path and Delivery contract.
Retention¶
How long a topic keeps a message after it is written, whether or not it was acked: at least one hour, or forever.
See: Manage topics.
Target¶
In a partition move, the node the partition should end up on. The assignment records it next to the current owner until the move completes.
See: Rebalance and decommission.
Topic¶
A named stream of messages that producers write to and consumers read from, split into partitions.
See: Core concepts.
Visibility timeout¶
How long a lease lasts, set for each topic when it is created (30 seconds by default). When it runs out, the message is delivered again.
See: Core concepts.