Skip to content

@scrapinghub Scrapinghub

Change the repository type filter

All

    Repositories list

    183 repositories

    • shub-workflow

      Public
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •15•13•3•2•Updated Feb 7, 2025Feb 7, 2025

    • web-poet

      Public
      Web scraping Page Objects core library
      pythonweb-scrapinghacktoberfestpage-objects
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •15•96•17•14•Updated Feb 7, 2025Feb 7, 2025

    • andi

      Public
      Library for annotation-based dependency injection
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •5•22•4•1•Updated Feb 7, 2025Feb 7, 2025

    • python-scrapinghub

      Public
      A client interface for Scrapinghub's API
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •63•204•24•2•Updated Feb 6, 2025Feb 6, 2025

    • price-parser

      Public
      Extract price amount and currency symbol from a raw text string
      hacktoberfest
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •50•320•17•9•Updated Feb 5, 2025Feb 5, 2025

    • frontera

      Public
      A scalable frontier for web crawlers
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •218•1.3k•79•19•Updated Feb 5, 2025Feb 5, 2025

    • extruct

      Public
      Extract embedded metadata from HTML markup
      microformatssemantic-webrdfajson-ldmicrodataopengraphhacktoberfest
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •113•878•38•14•Updated Feb 5, 2025Feb 5, 2025

    • hcf-backend

      Public
      Crawl Frontier HCF backend
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •5•7•2•2•Updated Feb 5, 2025Feb 5, 2025

    • dateparser

      Public
      python parser for human readable dates
      hacktoberfest
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •468•2.6k•290•50•Updated Feb 5, 2025Feb 5, 2025

    • scrapy-poet

      Public
      Page Object pattern for Scrapy
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •28•117•11•4•Updated Jan 29, 2025Jan 29, 2025

    • scrapinghub-stack-scrapy

      Public
      Software stack with latest Scrapy and updated deps
      Dockerfile
      •
      BSD 3-Clause "New" or "Revised" License
      •20•63•2•1•Updated Jan 6, 2025Jan 6, 2025

    • scrapinghub-entrypoint-scrapy

      Public
      Scrapy entrypoint for Scrapinghub job runner
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •16•25•8•1•Updated Jan 6, 2025Jan 6, 2025

    • spidermon

      Public
      Scrapy Extension for monitoring spiders execution.
      testingmonitoringscrapingcrawlingspidershacktoberfestmonitoring-toolscrapinghub
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •100•537•42•7•Updated Dec 10, 2024Dec 10, 2024

    • scrapy-frontera

      Public
      More flexible and featured Frontera scheduler for Scrapy
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •5•36•2•2•Updated Nov 29, 2024Nov 29, 2024

    • social-app-django

      Public
      Python Social Auth - Application - Django
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •383•2•0•1•Updated Nov 18, 2024Nov 18, 2024

    • Formasaurus

      Public
      Formasaurus tells you the type of an HTML form and its fields using machine learning
      HTML
      •48•7•1•0•Updated Nov 7, 2024Nov 7, 2024

    • number-parser

      Public
      Parse numbers written in natural language
      hacktoberfest
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •23•109•12•6•Updated Oct 23, 2024Oct 23, 2024

    • python-crfsuite

      Public
      A python binding for crfsuite
      data-sciencecrfcrfsuite
      Python
      •
      MIT License
      •222•772•45•3•Updated Oct 1, 2024Oct 1, 2024

    • streamparse

      Public
      streamparse lets you run Python code against real-time streams of data. Integrates with Apache Storm.
      Python
      •
      Apache License 2.0
      •218•2•0•1•Updated Sep 20, 2024Sep 20, 2024

    • splash

      Public
      Lightweight, scriptable browser as a service with an HTTP API
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •512•4.1k•376•26•Updated Aug 2, 2024Aug 2, 2024

    • pgcontents

      Public
      A Postgres-backed ContentsManager implementation for IPython
      Python
      •
      Apache License 2.0
      •85•2•0•1•Updated Jul 18, 2024Jul 18, 2024

    • shublang

      Public
      Pluggable DSL that uses pipes to perform a series of linear transformations to extract data
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •8•15•23•6•Updated Jul 9, 2024Jul 9, 2024

    • woodpecker

      Public
      An opinionated fork of the Drone CI system
      Go
      •
      Other
      •386•0•0•5•Updated Jul 7, 2024Jul 7, 2024

    • varanus

      Public
      A command line spider monitoring tool
      spidermonitoringpython36
      Python
      •7•8•2•2•Updated Jul 6, 2024Jul 6, 2024

    • scrapyrt

      Public
      HTTP API for Scrapy spiders
      pythoncrawlerscrapercrawlingtwistedscrapywebcrawlerhacktoberfestwebcrawlinghacktoberfest2021
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •161•847•24•6•Updated Jun 28, 2024Jun 28, 2024

    • portia

      Public
      Visual scraping for Scrapy
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1.4k•9.3k•111•19•Updated Jun 26, 2024Jun 26, 2024

    • sklearn-crfsuite

      Public
      scikit-learn inspired API for CRFsuite
      Python
      •215•2•0•0•Updated Jun 18, 2024Jun 18, 2024

    • webstruct-demo

      Public
      HTTP demo for https://github.com/scrapinghub/webstruct
      Python
      •
      MIT License
      •2•4•0•3•Updated Jun 17, 2024Jun 17, 2024

    • autologin

      Public
      A project to attempt to automatically login to a website given a single seed
      Python
      •
      Apache License 2.0
      •43•11•0•2•Updated Jun 17, 2024Jun 17, 2024

    • python-intercom

      Public
      Python wrapper for the Intercom API.
      Python
      •
      Other
      •146•1•0•1•Updated Jun 17, 2024Jun 17, 2024