Use Prefix key to filter objects at server side

Update doc strings and doc generation process

README.md

@@ -25,7 +25,7 @@
 [![pages-build-deployment][gha-pages-badge]][gha-pages]
 
 # S3 Download
-Python module to download all the objects in an S3 bucket.
+Python module to download objects in an S3 bucket.
 
 ### Installation
 ```shell

docs/README.html

@@ -61,7 +61,7 @@ <h3>Navigation</h3>
 <a class="reference external" href="https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment"><img alt="pages-build-deployment" src="https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment/badge.svg" /></a></p>
 <section id="s3-download">
 <h1>S3 Download<a class="headerlink" href="#s3-download" title="Permalink to this heading">¶</a></h1>
-<p>Python module to download all the objects in an S3 bucket.</p>
+<p>Python module to download objects in an S3 bucket.</p>
 <section id="installation">
 <h2>Installation<a class="headerlink" href="#installation" title="Permalink to this heading">¶</a></h2>
 <div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>pip<span class="w"> </span>install<span class="w"> </span>s3-downloader
@@ -98,11 +98,35 @@ <h2>Coding Standards<a class="headerlink" href="#coding-standards" title="Permal
 Clean code with pre-commit hooks: <a class="reference external" href="https://flake8.pycqa.org/en/latest/"><code class="docutils literal notranslate"><span class="pre">flake8</span></code></a> and
 <a class="reference external" href="https://pycqa.github.io/isort/"><code class="docutils literal notranslate"><span class="pre">isort</span></code></a></p>
 </section>
+<section id="release-notes">
+<h2><a class="reference external" href="https://github.com/thevickypedia/s3-downloader/blob/main/release_notes.rst">Release Notes</a><a class="headerlink" href="#release-notes" title="Permalink to this heading">¶</a></h2>
+<p><strong>Requirement</strong></p>
+<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>python<span class="w"> </span>-m<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>gitverse
+</pre></div>
+</div>
+<p><strong>Usage</strong></p>
+<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>gitverse-release<span class="w"> </span>reverse<span class="w"> </span>-f<span class="w"> </span>release_notes.rst<span class="w"> </span>-t<span class="w"> </span><span class="s1">&#39;Release Notes&#39;</span>
+</pre></div>
+</div>
+</section>
+<section id="linting">
+<h2>Linting<a class="headerlink" href="#linting" title="Permalink to this heading">¶</a></h2>
+<p><code class="docutils literal notranslate"><span class="pre">pre-commit</span></code> will ensure linting, run pytest, generate runbook &amp; release notes, and validate hyperlinks in ALL
+markdown files (including Wiki pages)</p>
+<p><strong>Requirement</strong></p>
+<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>pip<span class="w"> </span>install<span class="w"> </span><span class="nv">sphinx</span><span class="o">==</span><span class="m">5</span>.1.1<span class="w"> </span>pre-commit<span class="w"> </span>recommonmark
+</pre></div>
+</div>
+<p><strong>Usage</strong></p>
+<div class="highlight-shell notranslate"><div class="highlight"><pre><span></span>pre-commit<span class="w"> </span>run<span class="w"> </span>--all-files
+</pre></div>
+</div>
 <section id="pypi-package">
-<h2>Pypi Package<a class="headerlink" href="#pypi-package" title="Permalink to this heading">¶</a></h2>
+<h3>Pypi Package<a class="headerlink" href="#pypi-package" title="Permalink to this heading">¶</a></h3>
 <p><a class="reference external" href="https://packaging.python.org/tutorials/packaging-projects/"><img alt="pypi-module" src="https://img.shields.io/badge/Software%20Repository-pypi-1f425f.svg" /></a></p>
 <p><a class="reference external" href="https://pypi.org/project/s3-downloader/">https://pypi.org/project/s3-downloader/</a></p>
 </section>
+</section>
 <section id="license-copyright">
 <h2>License &amp; copyright<a class="headerlink" href="#license-copyright" title="Permalink to this heading">¶</a></h2>
 <p>© Vignesh Rao</p>
@@ -128,7 +152,11 @@ <h3><a href="index.html">Table of Contents</a></h3>
 </ul>
 </li>
 <li><a class="reference internal" href="#coding-standards">Coding Standards</a></li>
+<li><a class="reference internal" href="#release-notes">Release Notes</a></li>
+<li><a class="reference internal" href="#linting">Linting</a><ul>
 <li><a class="reference internal" href="#pypi-package">Pypi Package</a></li>
+</ul>
+</li>
 <li><a class="reference internal" href="#license-copyright">License &amp; copyright</a></li>
 </ul>
 </li>

docs/README.md

@@ -9,23 +9,23 @@
 
 **Repo Stats**
 
-[![GitHub](https://img.shields.io/github/license/thevickypedia/s3-downloader)](https://github.com/thevickypedia/s3-downloader/blob/main/LICENSE)
-[![GitHub repo size](https://img.shields.io/github/repo-size/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
-[![GitHub code size](https://img.shields.io/github/languages/code-size/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
+[![GitHub](https://img.shields.io/github/license/thevickypedia/s3-downloader)][license]
+[![GitHub repo size](https://img.shields.io/github/repo-size/thevickypedia/s3-downloader)][repo]
+[![GitHub code size](https://img.shields.io/github/languages/code-size/thevickypedia/s3-downloader)][repo]
 
 **Activity**
 
-[![GitHub Repo created](https://img.shields.io/date/1618966420)](https://api.github.com/repos/thevickypedia/s3-downloader)
-[![GitHub commit activity](https://img.shields.io/github/commit-activity/y/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
-[![GitHub last commit](https://img.shields.io/github/last-commit/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
+[![GitHub Repo created](https://img.shields.io/date/1618966420)][repo]
+[![GitHub commit activity](https://img.shields.io/github/commit-activity/y/thevickypedia/s3-downloader)][repo]
+[![GitHub last commit](https://img.shields.io/github/last-commit/thevickypedia/s3-downloader)][repo]
 
 **Build Status**
 
-[![pypi-publish](https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml/badge.svg)](https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml)
-[![pages-build-deployment](https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment)
+[![pypi-publish][gha-pypi-badge]][gha-pypi]
+[![pages-build-deployment][gha-pages-badge]][gha-pages]
 
 # S3 Download
-Python module to download all the objects in an S3 bucket.
+Python module to download objects in an S3 bucket.
 
 ### Installation
 ```shell
@@ -58,13 +58,49 @@ Styling conventions: [`PEP 8`](https://www.python.org/dev/peps/pep-0008/) <br>
 Clean code with pre-commit hooks: [`flake8`](https://flake8.pycqa.org/en/latest/) and
 [`isort`](https://pycqa.github.io/isort/)
 
+## [Release Notes][release-notes]
+**Requirement**
+```shell
+python -m pip install gitverse
+```
+
+**Usage**
+```shell
+gitverse-release reverse -f release_notes.rst -t 'Release Notes'
+```
+
+## Linting
+`pre-commit` will ensure linting, run pytest, generate runbook & release notes, and validate hyperlinks in ALL
+markdown files (including Wiki pages)
+
+**Requirement**
+```shell
+pip install sphinx==5.1.1 pre-commit recommonmark
+```
+
+**Usage**
+```shell
+pre-commit run --all-files
+```
+
 ### Pypi Package
-[![pypi-module](https://img.shields.io/badge/Software%20Repository-pypi-1f425f.svg)](https://packaging.python.org/tutorials/packaging-projects/)
+[![pypi-module][pypi-logo]][pypi-tutorials]
 
-[https://pypi.org/project/s3-downloader/](https://pypi.org/project/s3-downloader/)
+[https://pypi.org/project/s3-downloader/][pypi]
 
 ## License & copyright
 
 &copy; Vignesh Rao
 
-Licensed under the [MIT License](https://github.com/thevickypedia/s3-downloader/blob/main/LICENSE)
+Licensed under the [MIT License][license]
+
+[license]: https://github.com/thevickypedia/s3-downloader/blob/main/LICENSE
+[release-notes]: https://github.com/thevickypedia/s3-downloader/blob/main/release_notes.rst
+[pypi]: https://pypi.org/project/s3-downloader/
+[pypi-tutorials]: https://packaging.python.org/tutorials/packaging-projects/
+[pypi-logo]: https://img.shields.io/badge/Software%20Repository-pypi-1f425f.svg
+[repo]: https://api.github.com/repos/thevickypedia/s3-downloader
+[gha-pages-badge]: https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment/badge.svg
+[gha-pypi-badge]: https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml/badge.svg
+[gha-pages]: https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment
+[gha-pypi]: https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml

docs/_sources/README.md.txt

@@ -9,23 +9,23 @@
 
 **Repo Stats**
 
-[![GitHub](https://img.shields.io/github/license/thevickypedia/s3-downloader)](https://github.com/thevickypedia/s3-downloader/blob/main/LICENSE)
-[![GitHub repo size](https://img.shields.io/github/repo-size/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
-[![GitHub code size](https://img.shields.io/github/languages/code-size/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
+[![GitHub](https://img.shields.io/github/license/thevickypedia/s3-downloader)][license]
+[![GitHub repo size](https://img.shields.io/github/repo-size/thevickypedia/s3-downloader)][repo]
+[![GitHub code size](https://img.shields.io/github/languages/code-size/thevickypedia/s3-downloader)][repo]
 
 **Activity**
 
-[![GitHub Repo created](https://img.shields.io/date/1618966420)](https://api.github.com/repos/thevickypedia/s3-downloader)
-[![GitHub commit activity](https://img.shields.io/github/commit-activity/y/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
-[![GitHub last commit](https://img.shields.io/github/last-commit/thevickypedia/s3-downloader)](https://api.github.com/repos/thevickypedia/s3-downloader)
+[![GitHub Repo created](https://img.shields.io/date/1618966420)][repo]
+[![GitHub commit activity](https://img.shields.io/github/commit-activity/y/thevickypedia/s3-downloader)][repo]
+[![GitHub last commit](https://img.shields.io/github/last-commit/thevickypedia/s3-downloader)][repo]
 
 **Build Status**
 
-[![pypi-publish](https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml/badge.svg)](https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml)
-[![pages-build-deployment](https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment)
+[![pypi-publish][gha-pypi-badge]][gha-pypi]
+[![pages-build-deployment][gha-pages-badge]][gha-pages]
 
 # S3 Download
-Python module to download all the objects in an S3 bucket.
+Python module to download objects in an S3 bucket.
 
 ### Installation
 ```shell
@@ -58,13 +58,49 @@ Styling conventions: [`PEP 8`](https://www.python.org/dev/peps/pep-0008/) <br>
 Clean code with pre-commit hooks: [`flake8`](https://flake8.pycqa.org/en/latest/) and
 [`isort`](https://pycqa.github.io/isort/)
 
+## [Release Notes][release-notes]
+**Requirement**
+```shell
+python -m pip install gitverse
+```
+
+**Usage**
+```shell
+gitverse-release reverse -f release_notes.rst -t 'Release Notes'
+```
+
+## Linting
+`pre-commit` will ensure linting, run pytest, generate runbook & release notes, and validate hyperlinks in ALL
+markdown files (including Wiki pages)
+
+**Requirement**
+```shell
+pip install sphinx==5.1.1 pre-commit recommonmark
+```
+
+**Usage**
+```shell
+pre-commit run --all-files
+```
+
 ### Pypi Package
-[![pypi-module](https://img.shields.io/badge/Software%20Repository-pypi-1f425f.svg)](https://packaging.python.org/tutorials/packaging-projects/)
+[![pypi-module][pypi-logo]][pypi-tutorials]
 
-[https://pypi.org/project/s3-downloader/](https://pypi.org/project/s3-downloader/)
+[https://pypi.org/project/s3-downloader/][pypi]
 
 ## License & copyright
 
 &copy; Vignesh Rao
 
-Licensed under the [MIT License](https://github.com/thevickypedia/s3-downloader/blob/main/LICENSE)
+Licensed under the [MIT License][license]
+
+[license]: https://github.com/thevickypedia/s3-downloader/blob/main/LICENSE
+[release-notes]: https://github.com/thevickypedia/s3-downloader/blob/main/release_notes.rst
+[pypi]: https://pypi.org/project/s3-downloader/
+[pypi-tutorials]: https://packaging.python.org/tutorials/packaging-projects/
+[pypi-logo]: https://img.shields.io/badge/Software%20Repository-pypi-1f425f.svg
+[repo]: https://api.github.com/repos/thevickypedia/s3-downloader
+[gha-pages-badge]: https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment/badge.svg
+[gha-pypi-badge]: https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml/badge.svg
+[gha-pages]: https://github.com/thevickypedia/s3-downloader/actions/workflows/pages/pages-build-deployment
+[gha-pypi]: https://github.com/thevickypedia/s3-downloader/actions/workflows/python-publish.yml

docs/genindex.html

@@ -42,33 +42,20 @@ <h3>Navigation</h3>
 <h1 id="index">Index</h1>
 
 <div class="genindex-jumpbox">
- <a href="#_"><strong>_</strong></a>
- | <a href="#B"><strong>B</strong></a>
+ <a href="#B"><strong>B</strong></a>
  | <a href="#C"><strong>C</strong></a>
  | <a href="#D"><strong>D</strong></a>
  | <a href="#E"><strong>E</strong></a>
  | <a href="#F"><strong>F</strong></a>
  | <a href="#G"><strong>G</strong></a>
  | <a href="#I"><strong>I</strong></a>
- | <a href="#L"><strong>L</strong></a>
  | <a href="#M"><strong>M</strong></a>
+ | <a href="#N"><strong>N</strong></a>
  | <a href="#P"><strong>P</strong></a>
  | <a href="#R"><strong>R</strong></a>
  | <a href="#S"><strong>S</strong></a>
 
 </div>
-<h2 id="_">_</h2>
-<table style="width: 100%" class="indextable genindextable"><tr>
-  <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="index.html#s3.dumper.Downloader._downloader">_downloader() (s3.dumper.Downloader method)</a>
-</li>
-  </ul></td>
-  <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="index.html#s3.dumper.Downloader._get_objects">_get_objects() (s3.dumper.Downloader method)</a>
-</li>
-  </ul></td>
-</tr></table>
-
 <h2 id="B">B</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
@@ -88,7 +75,17 @@ <h2 id="C">C</h2>
 <h2 id="D">D</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="index.html#s3.logger.default_format">default_format() (in module s3.logger)</a>
+</li>
+      <li><a href="index.html#s3.logger.default_handler">default_handler() (in module s3.logger)</a>
+</li>
+  </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="index.html#s3.logger.default_logger">default_logger() (in module s3.logger)</a>
+</li>
       <li><a href="index.html#s3.dumper.Downloader">Downloader (class in s3.dumper)</a>
+</li>
+      <li><a href="index.html#s3.dumper.Downloader.downloader">downloader() (s3.dumper.Downloader method)</a>
 </li>
   </ul></td>
 </tr></table>
@@ -104,7 +101,7 @@ <h2 id="E">E</h2>
 <h2 id="F">F</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="index.html#s3.exceptions.InvalidDelimiter.format_error_message">format_error_message() (s3.exceptions.InvalidDelimiter method)</a>
+      <li><a href="index.html#s3.exceptions.InvalidPrefix.format_error_message">format_error_message() (s3.exceptions.InvalidPrefix method)</a>
 </li>
   </ul></td>
 </tr></table>
@@ -113,6 +110,10 @@ <h2 id="G">G</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="index.html#s3.dumper.Downloader.get_bucket_structure">get_bucket_structure() (s3.dumper.Downloader method)</a>
+</li>
+  </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="index.html#s3.dumper.Downloader.get_objects">get_objects() (s3.dumper.Downloader method)</a>
 </li>
   </ul></td>
 </tr></table>
@@ -124,15 +125,7 @@ <h2 id="I">I</h2>
 </li>
   </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="index.html#s3.exceptions.InvalidDelimiter">InvalidDelimiter</a>
-</li>
-  </ul></td>
-</tr></table>
-
-<h2 id="L">L</h2>
-<table style="width: 100%" class="indextable genindextable"><tr>
-  <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="index.html#s3.logger">logger (in module s3)</a>
+      <li><a href="index.html#s3.exceptions.InvalidPrefix">InvalidPrefix</a>
 </li>
   </ul></td>
 </tr></table>
@@ -147,11 +140,21 @@ <h2 id="M">M</h2>
         <li><a href="index.html#module-s3.dumper">s3.dumper</a>
 </li>
         <li><a href="index.html#module-s3.exceptions">s3.exceptions</a>
+</li>
+        <li><a href="index.html#module-s3.logger">s3.logger</a>
 </li>
       </ul></li>
   </ul></td>
 </tr></table>
 
+<h2 id="N">N</h2>
+<table style="width: 100%" class="indextable genindextable"><tr>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="index.html#s3.exceptions.NoObjectFound">NoObjectFound</a>
+</li>
+  </ul></td>
+</tr></table>
+
 <h2 id="P">P</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
@@ -184,13 +187,20 @@ <h2 id="S">S</h2>
         <li><a href="index.html#module-s3.dumper">module</a>
 </li>
       </ul></li>
-  </ul></td>
-  <td style="width: 33%; vertical-align: top;"><ul>
       <li>
     s3.exceptions
 
       <ul>
         <li><a href="index.html#module-s3.exceptions">module</a>
+</li>
+      </ul></li>
+  </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li>
+    s3.logger
+
+      <ul>
+        <li><a href="index.html#module-s3.logger">module</a>
 </li>
       </ul></li>
       <li><a href="index.html#s3.exceptions.S3Error">S3Error</a>