Skip to content

Commit

Permalink
chore: Add and use refs for API endpoints
Browse files Browse the repository at this point in the history
  • Loading branch information
jpmckinney committed Jul 12, 2024
1 parent 2b6264d commit c5c266b
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 12 deletions.
15 changes: 14 additions & 1 deletion docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ API

The following section describes the available resources in Scrapyd JSON API.

.. _daemonstatus.json:

daemonstatus.json
-----------------

Expand All @@ -24,6 +26,7 @@ Example response::

{ "status": "ok", "running": "0", "pending": "0", "finished": "0", "node_name": "node-name" }

.. _addversion.json:

addversion.json
---------------
Expand Down Expand Up @@ -53,7 +56,7 @@ schedule.json_ and listspiders.json_ allow you to explicitly set the desired pro

.. _packaging Version: https://packaging.pypa.io/en/stable/version.html

.. _scrapyd-schedule:
.. _schedule.json:

schedule.json
-------------
Expand Down Expand Up @@ -121,6 +124,8 @@ Example response::

{"status": "ok", "prevstate": "running"}

.. _listprojects.json:

listprojects.json
-----------------

Expand All @@ -137,6 +142,8 @@ Example response::

{"status": "ok", "projects": ["myproject", "otherproject"]}

.. _listversions.json:

listversions.json
-----------------

Expand All @@ -156,6 +163,8 @@ Example response::

{"status": "ok", "versions": ["r99", "r156"]}

.. _listspiders.json:

listspiders.json
----------------

Expand Down Expand Up @@ -224,6 +233,8 @@ Example response::

.. note:: All job data is kept in memory by default and will be reset when the Scrapyd service is restarted. See `jobstorage`_.

.. _delversion.json:

delversion.json
---------------

Expand All @@ -244,6 +255,8 @@ Example response::

{"status": "ok"}

.. _delproject.json:

delproject.json
---------------

Expand Down
2 changes: 1 addition & 1 deletion docs/deploy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
Deploying your project
======================

Deploying your project involves eggifying it and uploading the egg to Scrapyd via the `addversion.json <https://scrapyd.readthedocs.org/en/latest/api.html#addversion-json>`_ endpoint. You can do this manually, but the easiest way is to use the `scrapyd-deploy` tool provided by `scrapyd-client <https://github.com/scrapy/scrapyd-client>`_ which will do it all for you.
Deploying your project involves eggifying it and uploading the egg to Scrapyd via the `addversion.json <https://scrapyd.readthedocs.org/en/latest/api.html#addversion-json>`_ webservice. You can do this manually, but the easiest way is to use the `scrapyd-deploy` tool provided by `scrapyd-client <https://github.com/scrapy/scrapyd-client>`_ which will do it all for you.
20 changes: 10 additions & 10 deletions docs/news.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Added
~~~~~

- Add environment variables to override common options. See :doc:`config`.
- Add documentation on how to add endpoints. See :ref:`config-services`.
- Add documentation on how to add webservices (endpoints). See :ref:`config-services`.

Changed
~~~~~~~
Expand Down Expand Up @@ -62,7 +62,7 @@ Fixed
Added
~~~~~

- Add ``items_url`` and ``log_url`` to the response from the listjobs.json webservice. (@mxdev88)
- Add ``items_url`` and ``log_url`` to the response from the :ref:`listjobs.json` webservice. (@mxdev88)
- Scrapy 2.8 support. Scrapyd sets ``LOG_FILE`` and ``FEEDS`` command-line arguments, instead of ``SCRAPY_LOG_FILE`` and ``SCRAPY_FEED_URI`` environment variables.
- Python 3.11 support.
- Python 3.12 support. Use ``packaging.version.Version`` instead of ``distutils.LooseVersion``. (@pawelmhm)
Expand Down Expand Up @@ -94,7 +94,7 @@ Fixed
~~~~~

- Print Scrapyd's version instead of Twisted's version with ``--version`` (``-v``) flag. (@niuguy)
- Override Scrapy's ``LOG_STDOUT`` setting to ``False`` to suppress logging output for listspiders.json webservice. (@Lucioric2000)
- Override Scrapy's ``LOG_STDOUT`` setting to ``False`` to suppress logging output for :ref:`listspiders.json` webservice. (@Lucioric2000)

1.3.0 (2022-01-12)
------------------
Expand All @@ -103,9 +103,9 @@ Added
~~~~~

- Add support for HTTP authentication.
- Make ``project`` argument to listjobs.json optional, to easily query for all jobs.
- Make optional the ``project`` argument to the :ref:`listjobs.json` webservice, to easily query for all jobs.
- Improve HTTP headers across webservices.
- Add shortcut to jobs page to cancel a job using the cancel.json webservice.
- Add shortcut to jobs page to cancel a job using the :ref:`cancel.json` webservice.
- Add configuration options for job storage class and egg storage class.
- Improve test coverage.
- Python 3.7, 3.8, 3.9, 3.10 support.
Expand Down Expand Up @@ -147,10 +147,10 @@ Added

- Webservice

- Add daemonstatus.json webservice.
- Add project version argument to the schedule.json webservice.
- Add jobid argument to the schedule.json webservice.
- Add the run's PID to the response of the listjobs.json webservice.
- Add the :ref:`daemonstatus.json` webservice.
- Add a ``_version`` argument to the :ref:`schedule.json` webservice.
- Add a ``jobid`` argument to the :ref:`schedule.json` webservice.
- Add the run's PID to the response of the :ref:`listjobs.json` webservice.
- Include full tracebacks from Scrapy when failing to get spider list.
This makes debugging deployment problems easier, but webservice output noisier.

Expand Down Expand Up @@ -217,7 +217,7 @@ Added
~~~~~

- Add ``node_name`` (hostname) to webservice responses. (:commit:`fac3a5c`, :commit:`4aebe1c`)
- Add ``start_time`` to the response from the listjobs.json webservice. (:commit:`6712af9`, :commit:`acd460b`)
- Add ``start_time`` to the response from the :ref:`listjobs.json` webservice. (:commit:`6712af9`, :commit:`acd460b`)

Changed
~~~~~~~
Expand Down

0 comments on commit c5c266b

Please sign in to comment.