Merge branch 'main' into feat/addCustomModifierPython

Author: lukasheinrich

.devcontainer/devcontainer.json

@@ -0,0 +1,22 @@
+{
+	"name": "pyhf-devcontainer",
+	"build": {
+		"context": "..",
+		"dockerfile": "../docker/Dockerfile"
+	},
+	"features": {
+		"ghcr.io/devcontainers/features/git:1": {}
+	},
+
+	"customizations": {
+		"vscode": {
+			"settings": {
+				"python.defaultInterpreterPath": "/opt/venv/bin/python"
+			},
+			"extensions": [
+				"ms-python.python",
+				"ms-toolsai.jupyter"
+			]
+		}
+	}
+}

.dockerignore

@@ -0,0 +1,2 @@
+.nox
+.*cache

.github/ISSUE_TEMPLATE/~release-checklist.md

@@ -10,23 +10,40 @@ about: Checklist for core developers to complete as part of making a release
 * [ ] Migrate any unresolved Issues or PRs from the [release GitHub project board](https://github.com/scikit-hep/pyhf/projects/) to a new project board.
 * [ ] Verify that there is a release notes file for the release under [``docs/release-notes``](https://github.com/scikit-hep/pyhf/tree/main/docs/release-notes).
 * [ ] Verify that the release notes files correctly summarize all development changes since the last release.
-* [ ] Draft email to [``pyhf-announcements`` mailing list](https://groups.google.com/group/pyhf-announcements/subscribe) that summarizes the main points of the release notes and circulate it for development team approval.
 * [ ] Update the checklist Issue template in the [``.github/ISSUE_TEMPLATE``](https://github.com/scikit-hep/pyhf/tree/main/.github/ISSUE_TEMPLATE) directory if there are revisions.
-* [ ] Make a release to [TestPyPI][TestPyPI_pyhf] using the [workflow dispatch event trigger](https://github.com/scikit-hep/pyhf/actions/workflows/publish-package.yml).
-* [ ] Verify that the project README is displaying correctly on [TestPyPI][TestPyPI_pyhf].
 * [ ] Add any new use citations or published statistical models to the [Use and Citations page][citations_page].
-* [ ] Verify that the citations on the [Use and Citations page][citations_page] are up to date with their current [INSPIRE](https://inspirehep.net/) record.
-* [ ] Update the [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) GitHub Action used for deployment to TestPyPI and PyPI to the latest stable release.
+* [ ] Verify that the citations on the [Use and Citations page][citations_page] are up to date with their current [INSPIRE](https://inspirehep.net/) record. Checking the [Dimensions listing of publication citations](https://app.dimensions.ai/discover/publication?or_subset_publication_citations=pub.1135154020) can be helpful to catch citations that are now journal publications.
 * [ ] Update the ``codemeta.json`` file in the release PR if its requirements have updated.
+* [ ] Update the [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) GitHub Action used for deployment to TestPyPI and PyPI to the latest stable release.
+* [ ] Make a release to [TestPyPI][TestPyPI_pyhf] using the [workflow dispatch event trigger](https://github.com/scikit-hep/pyhf/actions/workflows/publish-package.yml).
+* [ ] Verify that the project README is displaying correctly on [TestPyPI][TestPyPI_pyhf].
+* [ ] Draft email to [``pyhf-announcements`` mailing list](https://groups.google.com/group/pyhf-announcements/subscribe) that summarizes the main points of the release notes and circulate it for development team approval.
 
 [TestPyPI_pyhf]: https://test.pypi.org/project/pyhf/
 [citations_page]: https://scikit-hep.org/pyhf/citations.html
 
-## Once Release PR is Merged
+## Create Release Tag
 
-* [ ] Watch the CI to ensure that the deployment to [PyPI](https://pypi.org/project/pyhf/) is successful.
-* [ ] Create a [GitHub release](https://github.com/scikit-hep/pyhf/releases) from the generated PR tag and copy the release notes published to the GitHub release page. The creation of the GitHub release triggers all other release related activities.
+For a video walkthrough consult the [``pyhf`` ``v0.7.1`` release recording](https://youtu.be/ZV20tr3EpTw) on YouTube.
+
+* [ ] Use the [bump version](https://github.com/scikit-hep/pyhf/actions/workflows/bump-version.yml) GitHub Actions workflow perform a [dry run](https://scikit-hep.org/pyhf/development.html#release-tags) of the bump version to the new release tag.
+* [ ] Check the annotated tag in the dry run workflow logs to make sure it looks correct.
+* [ ] If the dry run passes as expected, run the same workflow with the dry run option set to ``false`` to bump the release tag version and push the new tag back to GitHub.
+* [ ] Verify the release tag was pushed to the correct branch.
+* [ ] Verify the release tag commit has bumped the correct versions.
+* [ ] Watch the CI to verify all tag based jobs finish correctly.
+* [ ] Verify the release for the tag on [TestPyPI][TestPyPI_pyhf] looks correct.
+
+## After Release Tag Pushed To GitHub
+
+* [ ] Create a [GitHub release](https://github.com/scikit-hep/pyhf/releases) from the new release tag and copy the release notes published to the GitHub release page. The creation of the GitHub release triggers all other release related activities.
    - [ ] Before pasting in the release notes copy the changes that the GitHub bot has already queued up and pasted into the tag and place them in the "Changes" section of the release notes. If the release notes are published before these are copied then they will be overwritten and you'll have to add them back in by hand.
+   - [ ] Create a corresponding [announcement GitHub Discussion](https://github.com/scikit-hep/pyhf/discussions/categories/announcements) for the release.
+* [ ] Watch the CI to ensure that the deployment to [PyPI](https://pypi.org/project/pyhf/) is successful.
+* [ ] Verify Docker images with the correct tags have been deployed to all container image registries.
+   - [ ] [Docker Hub](https://hub.docker.com/r/pyhf/pyhf/tags)
+   - [ ] [OSG Harbor](https://hub.opensciencegrid.org/harbor/projects/866/repositories/pyhf/)
+   - [ ] [CERN Harbor](https://registry.cern.ch/harbor/projects/3550/repositories/pyhf/artifacts-tab)
 * [ ] Verify there is a new [Zenodo DOI](https://doi.org/10.5281/zenodo.1169739) minted for the release.
    - [ ] Verify that the new release archive metadata on Zenodo matches is being picked up as expected from [`CITATION.cff`](https://github.com/scikit-hep/pyhf/blob/main/CITATION.cff).
 * [ ] Verify that a Binder has properly built for the new release.
@@ -44,6 +61,5 @@ about: Checklist for core developers to complete as part of making a release
 * [ ] Update the [tutorial](https://github.com/pyhf/pyhf-tutorial) to use the new release number and API.
 * [ ] Make a PR to use the new release in the [CUDA enabled Docker images](https://github.com/pyhf/cuda-images).
 * [ ] Open a ticket on the CERN [Software Process and Infrastructure JIRA](https://sft.its.cern.ch/jira/browse/SPI) to update the version of `pyhf` available in the next LCG release.
-   - c.f. the [`v0.6.3` request ticket](https://sft.its.cern.ch/jira/browse/SPI-2086) as an example.
-* [ ] If the release is a **major** or **minor** release, open a [GitHub Release Radar](https://github.com/github/release-radar) Issue for the release to potentially get featured on GitHub's [Release Radar blog](https://github.blog/?s=release+radar).
+   - c.f. the [`v0.6.3` request ticket](https://sft.its.cern.ch/jira/browse/SPI-2086) and the [`v0.7.1` request ticket](https://sft.its.cern.ch/jira/browse/SPI-2333) as examples.
 * [ ] Close the [release GitHub Project board](https://github.com/scikit-hep/pyhf/projects/).

.github/workflows/dependencies-head.yml

@@ -64,7 +64,7 @@ jobs:
         python -m pip install --upgrade pip setuptools wheel
         python -m pip --no-cache-dir --quiet install --upgrade .[test]
         python -m pip uninstall --yes scipy
-        python -m pip install --upgrade --index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple scipy
+        python -m pip install --upgrade --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple scipy
         python -m pip list
 
     - name: Test with pytest
@@ -143,12 +143,13 @@ jobs:
         python -m pip install --upgrade pip setuptools wheel
         python -m pip --no-cache-dir --quiet install --upgrade .[test]
         python -m pip uninstall --yes matplotlib
-        # Need to use --extra-index-url as dependencies aren't on scipy-wheels-nightly package index.
+        # Need to use --extra-index-url as dependencies aren't on scientific-python-nightly-wheels package index.
         # Need to use --pre as dev releases will need priority over stable releases.
         python -m pip install \
           --upgrade \
           --pre \
-          --extra-index-url https://pypi.anaconda.org/scipy-wheels-nightly/simple \
+          --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple \
+          --extra-index-url https://pypi.org/simple/ \
           matplotlib
 
     - name: List installed Python packages

.github/workflows/docs.yml

@@ -78,9 +78,17 @@ jobs:
         [ ! -L "docs/_build/html/schemas" ]
         # is not empty
         [ "$(ls -A docs/_build/html/schemas)" ]
+        # is not empty
+        [ "$(ls -A docs/_build/html/lite)" ]
+
+    - name: Fix permissions if needed
+      run: |
+        chmod -c -R +rX "docs/_build/html/" | while read line; do
+          echo "::warning title=Invalid file permissions automatically fixed::$line"
+        done
 
     - name: Upload artifact
-      uses: actions/upload-pages-artifact@v1
+      uses: actions/upload-pages-artifact@v2
       with:
         path: 'docs/_build/html'
 

.github/workflows/merged.yml

@@ -15,5 +15,4 @@ jobs:
     - name: Trigger Binder build
       run: |
         # Use Binder build API to trigger repo2docker to build image on Google Cloud and Turing Institute Binder Federation clusters
-        bash binder/trigger_binder.sh https://gke.mybinder.org/build/gh/scikit-hep/pyhf/main
         bash binder/trigger_binder.sh https://turing.mybinder.org/build/gh/scikit-hep/pyhf/main

.github/workflows/publish-package.yml

@@ -25,8 +25,8 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  build-and-publish:
-    name: Build and publish Python distro to (Test)PyPI
+  build:
+    name: Build Python distribution
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v3
@@ -88,21 +88,48 @@ jobs:
     - name: List contents of wheel
       run: python -m zipfile --list dist/pyhf-*.whl
 
+    - name: Upload distribution artifact
+      uses: actions/upload-artifact@v3
+      with:
+        name: dist-artifact
+        path: dist
+
+  publish:
+    name: Publish Python distribution to (Test)PyPI
+    if: github.event_name != 'pull_request'
+    needs: build
+    runs-on: ubuntu-latest
+    # Mandatory for publishing with a trusted publisher
+    # c.f. https://docs.pypi.org/trusted-publishers/using-a-publisher/
+    permissions:
+      id-token: write
+    # Restrict to the environment set for the trusted publisher
+    environment:
+      name: publish-package
+
+    steps:
+    - name: Download distribution artifact
+      uses: actions/download-artifact@v3
+      with:
+        name: dist-artifact
+        path: dist
+
+    - name: List all files
+      run: ls -lh dist
+
     - name: Publish distribution 📦 to Test PyPI
       # Publish to TestPyPI on tag events of if manually triggered
       # Compare to 'true' string as booleans get turned into strings in the console
       if: >-
         (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v') && github.repository == 'scikit-hep/pyhf')
         || (github.event_name == 'workflow_dispatch' && github.event.inputs.publish == 'true' && github.repository == 'scikit-hep/pyhf')
-      uses: pypa/[email protected].5
+      uses: pypa/[email protected].8
       with:
-        password: ${{ secrets.test_pypi_password }}
         repository-url: https://test.pypi.org/legacy/
         print-hash: true
 
     - name: Publish distribution 📦 to PyPI
       if: github.event_name == 'release' && github.event.action == 'published' && github.repository == 'scikit-hep/pyhf'
-      uses: pypa/[email protected].5
+      uses: pypa/[email protected].8
       with:
-        password: ${{ secrets.pypi_password }}
         print-hash: true

.pre-commit-config.yaml

@@ -26,8 +26,8 @@ repos:
       # exclude generated files
       exclude: ^validation/|\.dtd$|\.xml$
 
-- repo: https://github.com/charliermarsh/ruff-pre-commit
-  rev: "v0.0.260"
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  rev: "v0.0.276"
   hooks:
     - id: ruff
       args: ["--fix", "--show-fixes"]
@@ -38,13 +38,13 @@ repos:
     - id: black-jupyter
 
 -   repo: https://github.com/asottile/blacken-docs
-    rev: 1.13.0
+    rev: 1.14.0
     hooks:
     - id: blacken-docs
       additional_dependencies: [black==23.3.0]
 
 -   repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.1.1
+    rev: v1.4.1
     # check the oldest and newest supported Pythons
     hooks:
       - &mypy
@@ -62,11 +62,11 @@ repos:
     rev: 1.7.0
     hooks:
     - id: nbqa-ruff
-      additional_dependencies: [ruff==0.0.260]
+      additional_dependencies: [ruff==0.0.276]
       args: ["--extend-ignore=F821,F401,F841,F811"]
 
 -   repo: https://github.com/codespell-project/codespell
-    rev: v2.2.4
+    rev: v2.2.5
     hooks:
     - id: codespell
       files: ^.*\.(py|md|rst)$

.zenodo.json

@@ -1,8 +1,8 @@
 {
     "description": "pure-Python HistFactory implementation with tensors and autodiff",
     "license": "Apache-2.0",
-    "title": "scikit-hep/pyhf: v0.7.1",
-    "version": "v0.7.1",
+    "title": "scikit-hep/pyhf: v0.7.2",
+    "version": "v0.7.2",
     "upload_type": "software",
     "creators": [
         {
@@ -36,7 +36,7 @@
     "related_identifiers": [
         {
             "scheme": "url",
-            "identifier": "https://github.com/scikit-hep/pyhf/tree/v0.7.1",
+            "identifier": "https://github.com/scikit-hep/pyhf/tree/v0.7.2",
             "relation": "isSupplementTo"
         }
     ]

CITATION.cff

@@ -14,11 +14,11 @@ authors:
   given-names: "Giordon"
   orcid: "https://orcid.org/0000-0001-6616-3433"
   affiliation: "SCIPP, University of California, Santa Cruz"
-title: "pyhf: v0.7.1"
-version: 0.7.1
+title: "pyhf: v0.7.2"
+version: 0.7.2
 doi: 10.5281/zenodo.1169739
-repository-code: "https://github.com/scikit-hep/pyhf/releases/tag/v0.7.1"
-url: "https://pyhf.readthedocs.io/en/v0.7.1/"
+repository-code: "https://github.com/scikit-hep/pyhf/releases/tag/v0.7.2"
+url: "https://pyhf.readthedocs.io/en/v0.7.2/"
 keywords:
   - python
   - physics

README.rst

@@ -1,4 +1,4 @@
-.. image:: https://raw.githubusercontent.com/scikit-hep/pyhf/main/docs/_static/img/pyhf-logo-small.png
+.. image:: https://raw.githubusercontent.com/scikit-hep/pyhf/main/docs/_static/img/pyhf-logo.svg
    :alt: pyhf logo
    :width: 320
    :align: center
@@ -309,11 +309,11 @@ the preferred BibTeX entry for citation of ``pyhf`` includes both the
 
    @software{pyhf,
      author = {Lukas Heinrich and Matthew Feickert and Giordon Stark},
-     title = "{pyhf: v0.7.1}",
-     version = {0.7.1},
+     title = "{pyhf: v0.7.2}",
+     version = {0.7.2},
      doi = {10.5281/zenodo.1169739},
      url = {https://doi.org/10.5281/zenodo.1169739},
-     note = {https://github.com/scikit-hep/pyhf/releases/tag/v0.7.1}
+     note = {https://github.com/scikit-hep/pyhf/releases/tag/v0.7.2}
    }
 
    @article{pyhf_joss,
@@ -365,22 +365,22 @@ and grant `OAC-1450377 <https://www.nsf.gov/awardsearch/showAward?AWD_ID=1450377
    :target: https://nsf.gov/awardsearch/showAward?AWD_ID=1836650
 .. |NumFOCUS Affiliated Project| image:: https://img.shields.io/badge/NumFOCUS-Affiliated%20Project-orange.svg?style=flat&colorA=E1523D&colorB=007D8A
    :target: https://numfocus.org/sponsored-projects/affiliated-projects
-.. |Docs from latest| image:: https://img.shields.io/badge/docs-v0.7.1-blue.svg
+.. |Docs from latest| image:: https://img.shields.io/badge/docs-v0.7.2-blue.svg
    :target: https://pyhf.readthedocs.io/
 .. |Docs from main| image:: https://img.shields.io/badge/docs-main-blue.svg
    :target: https://scikit-hep.github.io/pyhf
 .. |Jupyter Book tutorial| image:: https://jupyterbook.org/_images/badge.svg
    :target: https://pyhf.github.io/pyhf-tutorial/
 .. |Binder| image:: https://mybinder.org/badge_logo.svg
-   :target: https://mybinder.org/v2/gh/scikit-hep/pyhf/main?filepath=docs%2Fexamples%2Fnotebooks%2Fbinderexample%2FStatisticalAnalysis.ipynb
+   :target: https://mybinder.org/v2/gh/scikit-hep/pyhf/main?labpath=docs%2Fexamples%2Fnotebooks%2Fbinderexample%2FStatisticalAnalysis.ipynb
 
 .. |PyPI version| image:: https://badge.fury.io/py/pyhf.svg
    :target: https://badge.fury.io/py/pyhf
 .. |Conda-forge version| image:: https://img.shields.io/conda/vn/conda-forge/pyhf.svg
    :target: https://prefix.dev/channels/conda-forge/packages/pyhf
 .. |Supported Python versions| image:: https://img.shields.io/pypi/pyversions/pyhf.svg
    :target: https://pypi.org/project/pyhf/
-.. |Docker Hub pyhf| image:: https://img.shields.io/badge/pyhf-v0.7.1-blue?logo=Docker
+.. |Docker Hub pyhf| image:: https://img.shields.io/badge/pyhf-v0.7.2-blue?logo=Docker
    :target: https://hub.docker.com/r/pyhf/pyhf/tags
 .. |Docker Hub pyhf CUDA| image:: https://img.shields.io/badge/pyhf-CUDA-blue?logo=Docker
    :target: https://hub.docker.com/r/pyhf/cuda/tags

binder/postBuild

@@ -1,2 +1,5 @@
 python -m pip install --upgrade '.[all]'
-python -m pip install altair
+python -m pip install --upgrade \
+    ipywidgets \
+    ipympl \
+    altair

codemeta.json

@@ -122,5 +122,5 @@
         "runtimePlatform": "Python 3"
     },
     "url": "https://github.com/scikit-hep/pyhf",
-    "version": "0.7.1"
+    "version": "0.7.2"
 }

docker/Dockerfile

@@ -16,16 +16,47 @@ RUN apt-get -qq -y update && \
     python -m venv /usr/local/venv && \
     cd /code && \
     python -m pip --no-cache-dir install --upgrade pip setuptools wheel && \
-    python -m pip --no-cache-dir install .[xmlio,contrib] && \
+    python -m pip --no-cache-dir install '.[xmlio,contrib]' && \
     python -m pip list
 
 FROM base
+
+USER root
+
+SHELL [ "/bin/bash", "-c" ]
 ENV PATH=/usr/local/venv/bin:"${PATH}"
+
 RUN apt-get -qq -y update && \
     apt-get -qq -y install --no-install-recommends \
         curl && \
     apt-get -y autoclean && \
     apt-get -y autoremove && \
     rm -rf /var/lib/apt/lists/*
-COPY --from=builder /usr/local/venv /usr/local/venv
+
+# Create non-root user "moby" with uid 1000
+RUN adduser \
+      --shell /bin/bash \
+      --gecos "default user" \
+      --uid 1000 \
+      --disabled-password \
+      moby && \
+    chown -R moby /home/moby && \
+    mkdir /work && \
+    chown -R moby /work && \
+    echo -e "\nexport PATH=/usr/local/venv/bin:${PATH}\n" >> /home/moby/.bashrc
+
+COPY --from=builder --chown=moby /usr/local/venv /usr/local/venv/
+
+USER moby
+
+ENV USER ${USER}
+ENV HOME /home/moby
+WORKDIR ${HOME}/work
+
+# Use C.UTF-8 locale to avoid issues with ASCII encoding
+ENV LC_ALL=C.UTF-8
+ENV LANG=C.UTF-8
+
+ENV PATH=${HOME}/.local/bin:${PATH}
+
 ENTRYPOINT ["/usr/local/venv/bin/pyhf"]