Scrapy Introduction - Web Scraping With Python Using Scrapy

Web scraping is the process of scraping or extracting data from websites using programs or other tools. And in this video lesson, we will learn how to scrape websites using scrapy and implement the steps learned in the previous video with the same.

Full Course ✨ https://bit.ly/masterScrapy ✨ Web Scraping Masterclass

Again, we'll follow the steps discussed in the previous lesson (and if you haven't checked out the same, you can find the link below) :

With scrapy, we create a scrapy project. It's just like a web scraping project. In each project, we will create spiders ?️ because we will use them to crawl the web ?️.

The 1️⃣st step i.e. to send a request to the URL of the website, can be done with scrapy spiders start_urls or manually using just the URL. So, we provide the URL to scrapy and it will send a request for us.

The 2️⃣step i.e. handling the response is automatically done by scrapy. It downloads the HTML response of the website and provides that in a method under the spider, for further operations.

Now moving on to step 3️⃣ i.e. selecting the data. From our HTML response, using CSS Selectors or XPath, we can select them.

Now that we have selected the data we want from the HTML response, scrapy will grab the selected data and extract it out which is step 4️⃣.


⟪ Web Scraping For Beginners #3 ⟫
⏮️ #2 How Does Web Scraping Work? Step By Step [ https://youtu.be/wBjJHbNswV8 ]
⏭️ #4 Scrapy Installation - Windows [ https://youtu.be/JFg3Azcp3Gw ]

? Timestamps
00:00 Intro
00:12 Scrapy
00:30 Send Request
00:45 Get Response
01:01 Select Data
01:21 Extract Data

? Join the Community
Discord - https://discord.gg/ZqJEtKC7jU

? Intro Music
Music from Uppbeat (free for Creators!):
https://uppbeat.io/t/all-good-folks/work-it
License code: 7QUEOWCWQ3YNOLTI