Scrapy

Scrapy is a web scraping framework for Python that extracts structured data from websites. It's cross-platform, requires Python 3.10 or newer, and is maintained by Zyte (formerly Scrapinghub) along with a large community of contributors - making it one of the more established, long-running frameworks in the web-scraping space.

As a framework rather than a single-purpose tool, Scrapy handles the scaffolding around a scraper - making requests, following links, parsing responses, and exporting extracted data - so a developer writes the extraction logic specific to a given site rather than the crawling infrastructure around it.

Scrapy is a fit for Python developers building scrapers or crawlers who want an established, batteries-included framework, particularly for larger or long-running scraping projects where framework structure and maintainability matter.