Skip to content
@scrapy

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

Pinned Loading

  1. scrapy scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    Python 57.3k 10.9k

  2. scrapy.org scrapy.org Public

    The scrapy.org website

    HTML 64 143

Repositories

Showing 10 of 29 repositories
  • scrapy Public

    Scrapy, a fast high-level web crawling & scraping framework for Python.

    scrapy/scrapy’s past year of commit activity
    Python 57,340 BSD-3-Clause 10,937 459 (19 issues need help) 200 Updated Jun 28, 2025
  • itemloaders Public

    Library to populate items using XPath and CSS with a convenient API

    scrapy/itemloaders’s past year of commit activity
    Python 48 BSD-3-Clause 15 17 5 Updated Jun 28, 2025
  • itemadapter Public

    Common interface for data container classes

    scrapy/itemadapter’s past year of commit activity
    Python 68 BSD-3-Clause 12 5 2 Updated Jun 28, 2025
  • scrapy-bench Public

    A CLI for benchmarking Scrapy.

    scrapy/scrapy-bench’s past year of commit activity
    Python 31 MIT 15 6 1 Updated Jun 28, 2025
  • protego Public

    A pure-Python robots.txt parser with support for modern conventions.

    scrapy/protego’s past year of commit activity
    DIGITAL Command Language 69 BSD-3-Clause 28 7 (3 issues need help) 0 Updated Jun 24, 2025
  • flake8-scrapy Public

    A Flake8 plugin to catch common issues in Scrapy projects.

    scrapy/flake8-scrapy’s past year of commit activity
    Python 19 MIT 4 0 0 Updated Jun 18, 2025
  • sphinx-scrapy Public

    Sphinx extension for documentation in the Scrapy ecosystem

    scrapy/sphinx-scrapy’s past year of commit activity
    Python 1 BSD-3-Clause 0 0 0 Updated Jun 16, 2025
  • scrapyd-client Public

    Command line client for Scrapyd server

    scrapy/scrapyd-client’s past year of commit activity
    Python 775 BSD-3-Clause 146 5 0 Updated May 22, 2025
  • parsel Public

    Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

    scrapy/parsel’s past year of commit activity
    Python 1,236 BSD-3-Clause 151 32 (1 issue needs help) 12 Updated May 12, 2025
  • scrapy.org Public

    The scrapy.org website

    scrapy/scrapy.org’s past year of commit activity
    HTML 64 143 1 1 Updated May 8, 2025
View all repositories