Data Plane – Apache Kafka®'s Replication Protocol

https://cnfl.io/kafka-internals-101-module-3 | Apache Kafka® stores data durably and reliably, and its storage layer is different from the previous generation of systems in the messaging space where data is often missed or lost when a client isn't connected to the system.

In Kafka, multiple copies of data are maintained across a cluster to ensure data safety and achieve high availability and fault tolerance. Learn how Apache Kafka replication functions under the hood and how it contributes to higher durability and availability in this video featuring Jun Rao (Kafka Committer, PMC Member, VP of Kafka, and Co-Founder, Confluent). You will gain an understanding of the concepts behind leaders, followers, and replicas, as well as how replication is enabled and the role it plays in ensuring Kafka, delivers upon its data guarantees. This video is packed with details on writing events to Kafka, physical storage, the on-disk layout of data stored in Kafka, rebalancing data across brokers, replication strategy, in-sync replicas, observer replicas, replica lag, partition high watermark, and setting the replication factor.

Use the promo code INTERNALS101 to get $25 of free Confluent Cloud usage: https://www.confluent.io/confluent-cloud/tryfree?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-internals-101_content.apache-kafka

Promo code details: https://www.confluent.io/confluent-cloud-promo-disclaimer/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-internals-101_content.apache-kafka

LEARN MORE
► Hands-Free Kafka Replication: https://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-internals-101_content.apache-kafka
► Multi-region Clusters with Confluent Platform 5.4: https://www.confluent.io/blog/multi-region-data-replication/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-internals-101_content.apache-kafka
► Apache Kafka 101—Replication: https://developer.confluent.io/learn-kafka/apache-kafka/replication/?utm_source=youtube&utm_medium=video&utm_campaign=tm.devx_ch.cd-apache-kafka-internals-101_content.apache-kafka

ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion – designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.

#streamprocessing #apachekafka #kafka #confluent