Part 2 - A simple data pipeline using Python - Develop required functionality to read the data смотреть онлайн

Let us understand how to build an end-to-end pipeline using Python. Go through these videos to learn more about Pycharm, Git as well as setting up and validating the project to get data from a source MySQL Database to a target Postgres Database.

Here is the list of videos.

Video 1- Setup Project, GitHub Repository, and Develop Code to read the list of tables from a file: https://www.youtube.com/watch?v=BxLTTuLlvH0&list=PLf0swTFhTI8qhMM6uka63ASG1RrRHws4o&index=25
Video 2 (this one) - Develop required functionality to read the data from a table and generate insert statement using metadata: https://www.youtube.com/watch?v=czJ0j-9FK08&list=PLf0swTFhTI8qhMM6uka63ASG1RrRHws4o&index=26
Video 3 - Develop required functionality to write data to tables in target database: https://www.youtube.com/watch?v=V1nbPEhjLow&list=PLf0swTFhTI8qhMM6uka63ASG1RrRHws4o&index=28

If you are relatively new to Python feel free to follow our Mastering Python course at https://python.itversity.com

▶️Link for complete playlist - https://www.youtube.com/playlist?list=PLf0swTFhTI8pRV9DDzae2o1m-cqe5PtJ2

=============================================
More top rated #DataEngineering Courses from #itversity ??
_____________________________________________________
?Click below to get access to the course with one-month lab access for "Data Engineering Essentials Hands-on - SQL, Python, and Spark" -
???https://www.udemy.com/course/data-engineering-essentials-sql-python-and-spark/?referralCode=EEF55B4668DA42F6154D

?Data Engineering using AWS Analytics Services(Bestseller)
???https://www.udemy.com/course/data-engineering-using-aws-analytics-services/?referralCode=99ADF846582E1D7DAEA7

?Data Engineering using Databricks features on AWS and Azure (Highly Rated)
???https://www.udemy.com/course/data-engineering-using-databricks-on-aws-and-azure/?referralCode=EEA8219E6538F56E3B5B

?Data Engineering using Kafka and Spark Structured Streaming (NEW)
?https://www.udemy.com/course/data-engineering-using-kafka-and-spark-structured-streaming/?referralCode=30F204DEF4644FE9F112
============================================
TIMESTAMPS
00:00 – 02:52 Introduction
02:52 – 10:40 Connecting to the database
10:40 – 16:25 Establishing the connection & using keyword arguments
16:25 – 18:05 connect to the docker container & establish the connection
18:05 – 20:22 create the cursor
20:22 – 27:24 how to execute a query?
27:24 – 38:46 update our code to an extent where , our code can connecting to the Database start reading the data
38:46 – 40:20 print the column
40:20– 42:48 loading data get MySQL connection doubts & why use two connections difference between MySQL and postgreSQL
42:48– 50:00 connect the database & establish the connection
50:00 – 52:45 CRUD operations by inserting data into the postgres database
52:45 – 54:04 cycle of creating function for getting pg connection , build the insert query
54:04 – 01:11:00 load the data into the table
01:11:00 - 01:14:24 general introduction
01:14:24 – 01:16:45 Related query doubts

Connect with me or follow me at
https://www.linkedin.com/in/durga0gadiraju
https://www.facebook.com/itversity
https://github.com/dgadiraju
https://www.youtube.com/itversityin
https://twitter.com/itversity

#Python #pipeline #PythonProgramming #Data #itversity

Join this channel to get access to perks:
https://www.youtube.com/channel/UCakdSIPsJqiOLqylgoYmwQg/join