Kafka Consumer Groups CLI Demo | Kafka-Python

This video demonstrates the Kafka Consumer Groups concept using CLI

Prerequisite:
--------------------------
Consumer & Consumer Group in Kafka
https://youtu.be/QTi_oQhxARs

CLI Codes used
(this will only work if you installed kafka following the video:https://youtu.be/w6A-uDEb7JY)

Start Zookeeper:
---------------------------------------
F:/kafka_2.12-3.2.0/bin/windows/zookeeper-server-start.bat F:/kafka_2.12-3.2.0/config/zookeeper.properties

Start Kafka-server:
-----------------------------------------
F:/kafka_2.12-3.2.0/bin/windows/kafka-server-start.bat F:/kafka_2.12-3.2.0/config/server.properties

Create topic:
------------------------------------
F:/kafka_2.12-3.2.0/bin/windows/kafka-topics.bat --create --topic hello_world1 --bootstrap-server localhost:9092 --replication-factor 1 --partitions 3


Without Consumer Group:
--------------------------
F:/kafka_2.12-3.2.0/bin/windows/kafka-console-consumer.bat --topic hello_world1 --from-beginning --bootstrap-server localhost:9092


F:/kafka_2.12-3.2.0/bin/windows/kafka-console-consumer.bat --topic hello_world1 --from-beginning --bootstrap-server localhost:9092



Start Consumer with Group:
--------------------------------------------------------
F:/kafka_2.12-3.2.0/bin/windows/kafka-console-consumer.bat --topic hello_world1 --from-beginning --bootstrap-server localhost:9092 --group my-first-consumer-group

Producer Code:
--------------------------------
from time import sleep
from json import dumps
from kafka import KafkaProducer

topic_name='hello_world1'
producer = KafkaProducer(bootstrap_servers=['localhost:9092'],value_serializer=lambda x: dumps(x).encode('utf-8'))

while True:
message=input("Enter the message you want to send : ")
partition_no=int(input("In which partition you want to send ? "))
producer.send(topic_name, value=message,partition=partition_no)

producer.close()

Check this playlist for more Data Engineering related videos:
https://youtube.com/playlist?list=PLjfRmoYoxpNopPjdACgS5XTfdjyBcuGku

Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--
https://doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510

????????
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL