Apache Kafka is a versatile platform for handling real-time data streams. One essential task is producing messages to Kafka topics. In this article, we’ll explore how to produce Kafka messages using the command-line interface (CLI).
Prerequisites
Before you start, make sure you have the following:
- A running Kafka cluster.
- Kafka command-line tools installed on your system (part of the Kafka distribution).
Using the Kafka Console Producer
The Kafka distribution comes with a powerful command-line tool called kafka-console-producer that allows you to produce messages to Kafka topics. Here’s how you can use it:
kafka-console-producer.sh --bootstrap-server <kafka-broker>:<port> --topic <topic-name>
<kafka-broker>:<port>: Replace this with the address of your Kafka broker and the port it’s running on.<topic-name>: Specify the Kafka topic you want to produce messages to.
Once you run this command, you can start typing messages, and each line you enter will be treated as a separate message. Producing Messages with a Key
Kafka messages can have both a key and a value. You can produce messages with a key using the
--property "parse.key=true" --property "key.separator=:" flags.
kafka-console-producer.sh --bootstrap-server <kafka-broker>:<port> --topic <topic-name> --property "parse.key=true" --property "key.separator=:"
Now, when you enter messages, you can specify a key for each message by separating it with a colon (":").
Producing Messages with Avro Serialization
If your Kafka topic uses Avro serialization for messages, you can produce and serialize Avro messages using the –property value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer properties:
kafka-console-producer.sh --bootstrap-server <kafka-broker>:<port> --topic <topic-name> --property value.serializer=io.confluent.kafka.serializers.KafkaAvroSerializer --property schema.registry.url=<schema-registry-url>
<schema-registry-url>: Replace this with the URL of your Avro schema registry.
Conclusion
In this article, we’ve covered the essentials of producing messages to Kafka via the command-line interface (CLI). This approach is useful for quickly testing and sending messages to Kafka topics without writing custom producer code. The kafka-console-producer tool is a valuable utility that every Kafka developer should be familiar with.
