79108590

Date: 2024-10-21 04:55:22
Score: 1.5
Natty:
Report link

I've been looking for this information as well.

The best resource I have found that explains these steps in the context of a multi-node cluster is from Confluent:

https://docs.confluent.io/platform/current/installation/installing_cp/deb-ubuntu.html#ak-in-kraft-mode

Quoting relevant text from above doc to answer your questions...:

  1. Yes, generate one ID and use the same ID on all cluster nodes.

you must create a unique cluster ID and format the log directories with that ID...Before you start Kafka, you must use the kafka-storage tool with the random-uuid command to generate a cluster ID for each new cluster. You only need one cluster ID, which you will use to format each node in the cluster.

  1. You need to run the storage tool script on each node utilizing the same cluster ID:
    bin/kafka-storage.sh format -t <CLUSTER_ID> -c <path-to-server.properties>

use the cluster ID to format storage for each node in the cluster with the kafka-storage tool

  1. I'm not sure about your 3rd question regarding migration.

But with regards to your other meta-question: 'why they couldn't just automate it as part of boot up process'...

Previously, Kafka would format blank storage directories automatically and generate a new cluster ID automatically. One reason for the change is that auto-formatting can sometimes obscure an error condition. This is particularly important for the metadata log maintained by the controller and broker servers. If a majority of the controllers were able to start with an empty log directory, a leader might be able to be elected with missing committed data.

Reasons:
  • Blacklisted phrase (1): regards
  • Long answer (-1):
  • No code block (0.5):
  • Low reputation (1):
Posted by: fh8510