Some useful commands to manage Kafka

This post shows some useful commands to manage Kafka. (based on version 2.12)

Detailed information is on Kafka official documentation

Notice that some commands use zookeeper ip/port and others use kafka ip/port

Topic management commands

List all topics

${KAFKA_HOME}/bin/kafka-topics.sh –list –zookeeper zookeeper_ip:port

Create a topic

${KAFKA_HOME}/bin/kafka-topics.sh –create –zookeeper zookeeper_ip:port –replication-factor num –partitions num –topic topic_name

Get information for a topic

${KAFKA_HOME}/bin/kafka-topics.sh –describe –zookeeper zookeeper_ip:port –topic topic_name

Delete a topic

${KAFKA_HOME}/bin/kafka-topics.sh –delete –zookeeper zookeeper_ip:port –topic topic_name

Get offset for a topic

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.tools.GetOffsetShell –broker-list kafka_ip/port –topic topic_name –time time_option

• time_option : -1 (latest), -2 (earliest)

Consumer management commands

List all consumer groups registered on Kafka

${KAFKA_HOME}/bin/kafka-consumer-groups.sh –bootstrap-server kafka_ip/port –list

Get information for a consumer group

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.admin.ConsumerGroupCommand –bootstrap-server kafka_ip/port –group group_name –describe

Reset offset for a consumer group

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.admin.ConsumerGroupCommand –bootstrap-server kafka_ip/port –topic topic_name –group group_name –reset-offsets offset_option –execute

• offset_option : –to-earliest | –to-latest | –to-offset [offset_number]
• –to-earliest : the first offset after the consumer’s last committed offset, that is, not the beginning of the partition
• –to-latest : the latest offset of the partition
• –to-offset [offset_number] : offset number must be bigger thatn earliest offset

Delete a consumer group from the topic

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.admin.ConsumerGroupCommand –bootstrap-server kafka_ip/port –group group_name –delete