Skip to content

Latest commit

 

History

History
35 lines (23 loc) · 692 Bytes

README.md

File metadata and controls

35 lines (23 loc) · 692 Bytes

Setup Project

Dependencies

  1. pip install pymongo

  2. pip install scrapy

  3. pip install < Name of the Module if missing >

Run Scrapy RSS

Update Mongo Connection:

  1. Go to rss/rss/settings.py
  2. Change the Mongo URI as per your MONGO DB URI

To change crawl frequency :

  1. Go to rss/rss/settings.py]
  2. update CRAWL_TIME = 300 (In Seconds)

To Run the Scrapy

  1. Go to rss/spiders
  2. Run the command in terminal

python crawl.py

To update tag name of image

  1. docker-compose build
  2. docker-compose up
  3. docker tag rss-rss:latest pratikbtn/rss-rss:latest
  4. docker push pratikbtn/rss-rss

Reference

https://doc.scrapy.org/en/latest/index.html