Skip to content

Latest commit

 

History

History
27 lines (14 loc) · 1.38 KB

CONTRIBUTING.md

File metadata and controls

27 lines (14 loc) · 1.38 KB
Raw

Contributing

When websites change, spiders tend to break. I can't make a promise to keep this repository up-to-date all by myself, so pull requests are more than welcome!

Adding a new spider

  1. Create a branch for the spider, e.g. mtvarticle

  2. Add the spider name in pytest.ini and SPIDERS -file

  3. Add the spider in tests/test_spiders.py similar to others

  4. Add the spider API in finscraper/spiders.py

  5. Write the Scrapy spider under finscraper/scrapy_spiders by naming the spider exactly with the spider name, e.g. mtvarticle.py - use of flake8 linting and Google style docstrings is recommended

  6. Make sure the spider passes all non-benchmark tests within test_spiders.py

  7. Push your branch into Github and make a pull request against master

  8. Repository admin will merge to master and create a release after a review

Updating an existing spider

Steps 5. - 8. above.