Section 1
Getting started with Scrapy
Get started with scraping your first website
Section 2
Selecting data
Learn the various ways to selecting data on a web page
Section 3
Navigating pages
Learn to identify links and navigate from one page to another to continue scraping. Also learn how to selectively scrape patterns of urls to avoid scraping unnecessary pages.
Section 4
Exporting data
Export data to formats that can be parsed easily by other programs. Learn about the JSON Lines format
Section 5
Managing scraped data
Organize scraped data with objects and learn to post-process them with Item Pipelines.
Section 6
Configuring a spider
Learn to do more with a spider with some handy tricks.
Section 7
Advanced topics
Get up to speed with scrapy concepts that will come in handy.
Section 8
Ethical scraping
This section is a reminder that there are real people running websites and the things you can do to be polite when scraping.
Section 9
Workaround limitations
Learn how to get past limitations like forms, login or scrape websites with dynamic content
Section 10
Circumvent blocking
And then there are times that you need to fly below the radar
Section 11
Scraping other content
Learn to scrape different kinds of content using scrapy
Section 12
Deploying scrapy projects
Sure you can run your scraping projects on your own computer. But for large scraping projects, running them on servers allow you to scale beyond your bedroom
Section 13
Bonus content
Some tricks to speed up your implementation when scraping websites
From me to you
"Hello there.
Everything in this course is from my learnings in the last few years building and maintaining large scraping projects, and helping other folks do the same.
This course is all my notes and learnings, so that you move onto your projects without struggle."
— Akash Manohar (@HashNuke)