It seems the prometheus that comes with Milvus is enough to monitor my Kafka cluster. If I see need for more I will do a follow up.

In the monitoring panel above, you can see all the monitoring metrics for consume and create topic and many other operations. Kafka and Pulsar are very mature and not usually a bottleneck in Milvus clusters so it is usually not a top concern. If you already have an enterprise Pulsar or Kafka cluster, use your same tools (or even same cluster).

Note that if you are looking at specific tools, there are many more for Apache Kafka then Apache Pulsar.

Resources

Prometheus https://milvus.io/docs/monitor.md

Kafka UI https://github.com/provectus/kafka-ui

Pulsar CLI https://pulsar.apache.org/docs/next/deploy-monitoring/

Milvus Documentation https://milvus.io/docs/monitor_overview.md#Milvus-monitoring-framework-overview

Monitoring Kafka with Prometheus and Grafana https://medium.com/@rramiz.rraza/kafka-metrics-integration-with-prometheus-and-grafana-14fe318fbb8b

Monitor Kafka with Grafana https://grafana.com/solutions/kafka/monitor/