Introduction to Kafka Stream Processing in Python using Faust

Faust is a stream processing library, porting the ideas from Kafka Streams to Python.
This video explains , how to use Kafka (very popular distributed messaging queue) and Faust ( a python based distributed stream processing library) to create a simple stream processing application.

Faust Documentation Link: https://faust.readthedocs.io/en/latest/

Code:
--------
Start Zookeeper:
------------------------------
F:/kafka_2.12-3.3.1/bin/windows/zookeeper-server-start.bat F:/kafka_2.12-3.3.1/config/zookeeper.properties

Start Broker:
------------------------------
F:/kafka_2.12-3.3.1/bin/windows/kafka-server-start.bat F:/kafka_2.12-3.3.1/config/server.properties

Start Kafka Topic:
------------------------------
F:/kafka_2.12-3.3.1/bin/windows/kafka-topics.bat --create --topic hello_world --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1

Install the required modules:
------------------------------
pip install faust-streaming
pip install kafka-python

Python Code:
------------------------------
import faust

app=faust.App('demo-streaming',broker='localhost:9092')

topic = app.topic('hello_world', value_type=str,value_serializer='raw')

@app.agent(topic)
async def processor(stream):
async for message in stream:
print(f'Received {message}')

To run Fause Application
------------------------------
faust -A main worker -l info


Kafka Console Producer:
-------------------------------
F:/kafka_2.12-3.3.1/bin/windows/kafka-console-producer.bat --topic hello_world --bootstrap-server localhost:9092

Check this playlist for more Data Engineering related videos:
https://youtube.com/playlist?list=PLjfRmoYoxpNopPjdACgS5XTfdjyBcuGku

Apache Kafka form scratch
https://youtube.com/playlist?list=PLjfRmoYoxpNrs0VmIq6mOTqXP52RfZdRf

Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--
https://doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510

????????
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL