Add Jupyter Book documentation and CI pipeline (#146)

* Add Jupyter Book documentation and CI pipeline

* Fix Jupyter Book package conflict

* Update code documentation

* Update code documentation

* Update CADLabs logo

* Update GitHub action to only execute on main branch

.github/workflows/jupyter-book.yml

@@ -0,0 +1,61 @@
+name: deploy-book
+
+# Run this when the master or main branch changes
+on:
+  branchs: [ main ]
+  # push:
+  #   branches: [ main ]
+  # pull_request:
+  #   branches: [ main ]
+  # If your git repository has the Jupyter Book within some-subfolder next to
+  # unrelated files, you can make this run only if a file within that specific
+  # folder has been modified.
+  #
+  # paths:
+  # - some-subfolder/**
+
+# This job installs dependencies, builds the book, and pushes it to `gh-pages`
+jobs:
+  deploy-book:
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    steps:
+    - uses: actions/checkout@v3
+
+    # Install dependencies
+    - name: Set up Python 3.9
+      uses: actions/setup-python@v4
+      with:
+        python-version: 3.9
+
+    - name: Install dependencies
+      run: |
+        pip install -r requirements.txt
+
+    # (optional) Cache your executed notebooks between runs
+    # if you have config:
+    # execute:
+    #   execute_notebooks: cache
+    - name: Cache executed notebooks
+      uses: actions/cache@v3
+      with:
+        path: docs/_build/.jupyter_cache
+        key: jupyter-book-cache-${{ hashFiles('requirements.txt') }}
+
+    # Build the book
+    - name: Build the book
+      run: |
+        make build-docs
+
+    # Upload the book's HTML as an artifact
+    - name: Upload artifact
+      uses: actions/upload-pages-artifact@v2
+      with:
+        path: "docs/_build/html"
+
+    # Deploy the book's HTML to GitHub Pages
+    - name: Deploy to GitHub Pages
+      id: deployment
+      uses: actions/deploy-pages@v2

.gitignore

@@ -106,13 +106,13 @@ celerybeat.pid
 *.sage.py
 
 # Environments
-.env
-.venv
-env/
-venv/
-ENV/
-env.bak/
-venv.bak/
+.env*
+.venv*
+env*/
+venv*/
+ENV*/
+env*.bak/
+venv*.bak/
 
 # Spyder project settings
 .spyderproject
@@ -142,6 +142,7 @@ scratchpad/
 *.nbconvert.ipynb
 *.dat
 drafts/
+_autosummary/
 
 # Masterclass exam
 **/*answers.ipynb

Makefile

@@ -20,15 +20,14 @@ test: execute-notebooks
 	# Run Pytest tests
 	python3 -m pytest -m "not api_test" tests
 
-build-docs: docs-pdoc docs-jupyter-book
+build-docs: docs-jupyter-book
 
 docs-pdoc:
 	pdoc --html model -o docs --force
 
 docs-jupyter-book:
 	jupyter-book clean docs
-	jupyter-book build --config docs/_config.yml --toc docs/_toc.yml --path-output docs .
-	cp -r ./docs/model ./docs/_build/html/docs/model
+	python build_docs.py
 
 serve-docs:
 	gunicorn -w 4 -b 127.0.0.1:5000 docs.server:app

build_docs.py

@@ -0,0 +1,5 @@
+from jupyter_book.cli.main import main
+
+# See https://stackoverflow.com/questions/74367920/sphinx-recursive-autosummary-not-importing-modules
+# jupyter-book build --config docs/_config.yml --toc docs/_toc.yml --path-output docs .
+main(["build", "--config", "docs/_config.yml", "--toc", "docs/_toc.yml", "--path-output", "docs", "."])

docs/_config.yml

@@ -1,9 +1,27 @@
-title: "CADLabs Ethereum Research Model Documentation"
-# logo: media/_.png
+title: "CADLabs Ethereum Economic Model"
+logo: docs/logo.png
 only_build_toc_files: true
 execute:
-  execute_notebooks: "off"
+  # execute_notebooks: off
+  execute_notebooks: cache
+parse:
+  myst_enable_extensions:
+    # don't forget to list any other extensions you want enabled,
+    # including those that are enabled by default!
+    - amsmath
+    - dollarmath
+    - html_image
+    - attrs_inline
 sphinx:
+  extra_extensions:
+  - 'sphinx.ext.autodoc'
+  - 'sphinx.ext.autosummary'
+  - 'sphinx.ext.napoleon'
+  - 'sphinx.ext.viewcode'
   config:
+    # html_theme: pydata_sphinx_theme
+    templates_path: ['docs/_templates/autosummary']
+    add_module_names: True
+    autosummary_generate: True
     html_js_files:
-    - https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js
+    - https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js

docs/_templates/autosummary/custom-class-templates.rst

@@ -0,0 +1,32 @@
+{{ fullname | escape | underline}}
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+   :members:                             
+   :show-inheritance:                         
+   :inherited-members:                          
+
+   {% block methods %}
+   .. automethod:: __init__
+
+   {% if methods %}
+   .. rubric:: {{ _('Methods') }}
+
+   .. autosummary::
+   {% for item in methods %}
+      ~{{ name }}.{{ item }}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}
+
+   {% block attributes %}
+   {% if attributes %}
+   .. rubric:: {{ _('Attributes') }}
+
+   .. autosummary::
+   {% for item in attributes %}
+      ~{{ name }}.{{ item }}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}

docs/_templates/autosummary/custom-module-template.rst

@@ -0,0 +1,66 @@
+{{ fullname | escape | underline}}
+
+.. automodule:: {{ fullname }}
+
+   {% block attributes %}
+   {% if attributes %}
+   .. rubric:: Module Attributes
+
+   .. autosummary::
+      :toctree:                                          
+   {% for item in attributes %}
+      {{ item }}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}
+
+   {% block functions %}
+   {% if functions %}
+   .. rubric:: {{ _('Functions') }}
+
+   .. autosummary::
+      :toctree:                                     
+   {% for item in functions %}
+      {{ item }}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}
+
+   {% block classes %}
+   {% if classes %}
+   .. rubric:: {{ _('Classes') }}
+
+   .. autosummary::
+      :toctree:                            
+      :template: custom-class-template.rst          
+   {% for item in classes %}
+      {{ item }}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}
+
+   {% block exceptions %}
+   {% if exceptions %}
+   .. rubric:: {{ _('Exceptions') }}
+
+   .. autosummary::
+      :toctree:                                    
+   {% for item in exceptions %}
+      {{ item }}
+   {%- endfor %}
+   {% endif %}
+   {% endblock %}
+
+{% block modules %}
+{% if modules %}
+.. rubric:: Modules
+
+.. autosummary::
+   :toctree:
+   :template: custom-module-template.rst  
+   :recursive:
+{% for item in modules %}
+   {{ item }}
+{%- endfor %}
+{% endif %}
+{% endblock %}

docs/_toc.yml

@@ -1,13 +1,19 @@
 format: jb-book
-root: docs/landing-page
+root: README
 parts:
+  - caption: Model Overview
+    chapters:
+    # - file: README
+    #   title: "Model README"
+    - file: ASSUMPTIONS
+    - file: ROADMAP
+    - file: CHANGELOG
   - caption: Model Specification
     chapters:
     - file: docs/model_specification/mathematical_specification
   - caption: Documentation
     chapters:
-    - url: https://cadlabs.gitlab.io/cadlabs-ethereum-model-docs/docs/index.html
-      title: Model Code Documentation
+    - file: docs/code.rst
   - caption: Experiment Notebooks
     chapters:
     - file: experiments/notebooks/0_README

docs/code.rst

@@ -0,0 +1,15 @@
+Model Code Documentation
+=================
+
+.. autosummary::
+  :toctree: _autosummary
+  :recursive:
+
+  model.parts
+  model.constants
+  model.state_update_blocks
+  model.state_variables
+  model.stochastic_processes
+  model.system_parameters
+  model.types
+  model.utils

docs/introduction.md

@@ -0,0 +1,21 @@
+# Introduction
+
+A modular dynamical-systems model of Ethereum's validator economics, based on the open-source Python library [radCAD](https://github.com/CADLabs/radCAD), an extension to [cadCAD](https://cadcad.org).
+
+This open-source model was developed in collaboration with the Ethereum Robust Incentives Group and funded by an Ethereum ESP (Ecosystem Support Program) grant. While originally scoped with purely modelling-educational intent as part of the cadCAD Edu online course "[cadCAD Masterclass: Ethereum Validator Economics](https://www.cadcad.education/course/masterclass-ethereum)", it has evolved to become a highly versatile, customizable and extensible research model that includes a list of [model extension ideas](#Model-Extension-Roadmap). The model is focused on epoch- and population-level Ethereum validator economics across different deployment types and – at least in its initial setup – abstracts from slot- and agent-level dynamics. Please see [Model Assumptions](ASSUMPTIONS.md) for further context.
+
+* GitHub repo: [CADLabs Ethereum Economic Model](https://github.com/CADLabs/ethereum-economic-model)
+* Latest model release version: [Subgraph / v1.1.7](https://github.com/CADLabs/ethereum-economic-model/releases/tag/v1.1.7)
+* Implements the official Ethereum [Altair](https://github.com/ethereum/eth2.0-specs#altair) spec updates in the [Blue Loop / v1.1.0-alpha.7](https://github.com/ethereum/eth2.0-specs/releases/tag/v1.1.0-alpha.7) release
+
+## Model Features
+
+* Configurable to reflect protocol behaviour at different points in time of the development roadmap (referred to as "upgrade stages"):
+  * Post Beacon Chain launch, pre EIP-1559, pre PoS (validators receive PoS incentives, EIP-1559 disabled, and PoW still in operation)
+  * Post Beacon Chain launch, post EIP-1559, pre PoS (validators receive PoS incentives, EIP-1559 enabled with miners receiving priority fees, and PoW still in operation)
+  * Post Beacon Chain launch, post EIP-1559, post PoS (validators receive PoS incentives, EIP-1559 enabled with validators receiving priority fees, and PoW deprecated)
+* Flexible calculation granularity: By default, State Variables, System Metrics, and System Parameters are calculated at epoch level and aggregated daily (~= 225 epochs). Users can easily change epoch aggregation using the delta-time (`dt`) parameter. The model can be extended for slot-level granularity and analysis if that is desired (see [Model Extension Roadmap](#Model-Extension-Roadmap)).
+* Supports [state-space analysis](https://en.wikipedia.org/wiki/State-space_representation) (i.e. simulation of system state over time) and [phase-space analysis](https://en.wikipedia.org/wiki/Phase_space) (i.e. generation of all unique system states in a given experimental setup).
+* Customizable processes to set important variables such as ETH price, ETH staked, and EIP-1559 transaction pricing.
+* Modular model structure for convenient extension and modification. This allows different user groups to refactor the model for different purposes, rapidly test new incentive mechanisms, or update the model as Ethereum implements new protocol improvements.
+* References to official [Eth2 specs](https://github.com/ethereum/eth2.0-specs) in Policy and State Update Function logic. This enables seamless onboarding of protocol developers and allows the more advanced cadCAD user to dig into the underlying protocol design that inspired the logic.