Published 5/2023
MP4 | Video: h264, 1280×720 | Audio: AAC, 44.1 KHz
Language: English | Size: 866.77 MB | Duration: 2h 15m
Build End-to-End Real-Time Streaming Pipelines with Kafka, Flink, Elasticsearch & Kibana in Python (last versions)
What you’ll learn
Master data ingestion: Efficiently process diverse data streams.
Unleash Kafka’s power: Understand core concepts for optimal streaming.
Learn major CLIs: kafka-topics, kafka-console-producer, kafka-console-consumer…
Python + Kafka: Build practical skills for producers, consumers, Topics, Partitions, Brokers, and more.
Create your Producers and Consumers in Python to interact with Kafka
Real-world Twitter data: Ingest and scale data seamlessly into HDFS.
Streamline with Kafka and Flink: Craft end-to-end pipelines for real-time analytics.
Real-time Analytics and Visualization: Master the art of analyzing streaming data and visualizing insights using Elasticsearch and Kibana in your Kafka pipeline
Requirements
Some understanding of Python Programming
Good to have knowledge about Linux command line
Desire to Master Big Data Streaming
Good to have knowledge on big data processing with Flink or Spark
Description
Unleashing the Power of Apache Kafka and Flink: Cutting-Edge Hands-on Experience with real life case studiesThis is the only updated Big Data Streaming Course using Kafka with Flink in python ! (Course newly recorded with Kafka 3.+, Flink 1.14.4, ES 7.17.7) Discover the unrivaled potential of Apache Kafka and the hidden gem of data processing, Flink, in our dynamic course. While Flink may be lesser-known than Spark, it’s a powerful tool that surpasses its counterparts in certain aspects.We’ll dive deep into Kafka’s core concepts, equipping you with the knowledge to build robust streaming pipelines. But we won’t stop there – we’ll showcase Flink’s prowess as we explore real-time data processing and analytics.Rest assured, all hands-on exercises are meticulously crafted using the latest versions of Kafka and Flink. Forget about outdated code or compatibility issues – we ensure you’re working with cutting-edge tools, ready to conquer the real world.Although Flink may have a smaller community compared to Spark, this presents a unique opportunity for you to become an early adopter and join the pioneering minds pushing the boundaries of streaming analytics.We’ll guide you step-by-step as you build a complete streaming pipeline that captures live Twitter data, processes it in real-time, and unlocks valuable insights. With our carefully crafted exercises, you’ll gain practical experience in ingesting, transforming, and analyzing Twitter data using the latest versions of Kafka and Flink.Imagine harnessing the pulse of social media to gain actionable insights, all in real-time. From sentiment analysis to trending topics, you’ll explore the limitless possibilities of Twitter data analytics.So, step into the future of stream data processing with Kafka and Flink. Enroll now to gain an edge in the industry, with hands-on expertise on the latest versions of these powerful tools. Don’t miss out on this transformative learning experience – the world of real-time data awaits!
Overview
Section 1: Introduction
Lecture 1 Introduction
Section 2: Introduction to Data Ingestion
Lecture 2 Introduction & problematic of Data Ingestion
Lecture 3 what is Data Ingestion
Lecture 4 Data Ingestion Tools
Section 3: Introduction to Apache Kafka
Lecture 5 Introduction to Kafka – Publish Subscribe Architecture
Lecture 6 Advantages of Apache Kafka
Section 4: Kafka fundamentals
Lecture 7 Overview of Kafka components
Lecture 8 kafka topic
Lecture 9 Kafka Partitions
Lecture 10 Kafka Topic Replication
Section 5: Kafka CLI hands-on
Lecture 11 Kafka CLI Hands-on | Introduction
Lecture 12 Start 3 nodes Kafka Cluster
Lecture 13 Create Kafka topic with CLI
Lecture 14 Delete Kafka Topic with CLI
Lecture 15 Kafka Console CLI: Producer and Consumer
Lecture 16 Linux commands used for 3 nodes Kafka Cluster
Section 6: Kafka Hands-on in python
Lecture 17 Kafka Producer and Consumer Hands-on in python | Introduction
Lecture 18 Simple Kafka Producer in Python – Part 1 (coding)
Lecture 19 Kafka Producer in Python – Part 2 – ( testing)
Lecture 20 Kafka Consumer in python -Part 1 – (coding)
Lecture 21 Kafka Consumer in python -Part 2 – (running)
Lecture 22 Complete Kafka (Producer + Consumer) python code resources
Section 7: Real World Big Data Ingestion Python Project: Streaming Twitter Data with Kafka
Lecture 23 The architecture Design of the hands-on project
Lecture 24 Real time Data Source: Twitter API from Developer Platform
Lecture 25 Extracting Twitter Data Stream from API in python
Lecture 26 Create a Tweets Data Kafka Producer
Lecture 27 Create a Tweets Data Kafka Consumer
Lecture 28 Kafka Consumer : Store Data in Hadoop HDFS
Lecture 29 Complete Python Code resources (Twitter Producer & Consumer + HDFS Consumer)
Section 8: Real Time Streaming pipeline Handson : Kafka, Flink, ElasticSearch and Kibana
Lecture 30 Real Time Streaming Architecture Design: Kafka , Flink, Elasticsearch, & Kibana
Lecture 31 Requirement for this project (updated versions 3.+.+)
Lecture 32 Apache Flink Introduction | Apache Spark -VS- Flink | PyFlink
Lecture 33 Configure Flink to consume data from a Kafka topic as a data source
Lecture 34 Configure Flink to write the processed data to a Elasticsearch sink
Lecture 35 Real Time Tweets Word Count with pyFlink and Kafka
Lecture 36 Complete Python Code : Streaming pipeline
Section 9: Real World Project Exercice-Solution
Developers who want to learn the Data Ingestion, Apache Kafka , Streaming with Apache Flink,Big Data Architects who want to understand how Apache Kafka fits into their solution architecture,Those desiring to build robust streaming pipelines for real-time analytics,Big Data enthusiasts and software developers looking to expand their skill set,Professionals aiming to stay ahead in the rapidly evolving Big Data landscape,Data analysts and professionals in the field of real-time data processing who want to leverage Kafka and Flink for advanced analytics.,Beginners seeking to enter the world of Kafka and Flink streaming and gain practical hands on skills.,Aspiring data engineers and software developers eager to master Kafka, Flink, Elasticsearch, and Kibana for end-to-end real-time analytics pipelines.,Professionals aiming to stay ahead in the dynamic data landscape by acquiring comprehensive skills in real-time data processing and visualization.
Password/解压密码caxfwz
请先 ！