v0.3.0

What's Changed

New Feature: Inline data updates

You can now push a data update via opal that already includes the data itself (inline) instead of including a reference url and incurring another network round trip. This is best for small incremental data updates.

• Allow embedded data in DataUpdate by @orishavit in #294
• Minor bug fixes to inline data updates by @asafc in #306

Fixes: Performance and stability improvements to OPAL Scopes

OPAL Scopes is a new experimental feature that allows a single OPAL server to track multiple git repositories, and in essence enables true multi-tenancy by a single OPAL deployment. This release includes major improvements to the performance and stability of this feature.

• Improve opal scopes performance by @asafc in #298

Minor bug fixes

• Add redis dependency to OPAL by @orishavit in #296
• Update OPAL Server from worker also after initial clone by @orishavit in #297
• Fix OPAL server trying to use the broadcaster when it's not configured by @shaulk in #295

Documentation

• Publishing OPAL as online docusaurus documentation by @filipermit in #291
• Dependancy fix and removal of unecessary pages by @filipermit in #292
• Fixed transitive dependancies for docusaurus packages by @filipermit in #293
• Updated the navigation title to a more meaningful version by @filipermit in #302
• Permit.io link in the footer of docs. by @filipermit in #303

New Contributors

• @filipermit made their first contribution in #291
• @shaulk made their first contribution in #295

Full Changelog: 0.2.1...0.3.0

v0.2.1

What's Changed

• Add Scopes guide by @orishavit in #290
• Bugfix in makefile: fixed bug that caused import issues with the python packages (while the prebuilt docker images were working correctly)

Full Changelog: 0.2.0...0.2.1

v0.2.0

What's Changed

TL;DR

This version has a lot of small fixes and improvements as longs as a few new features: restricted topics, nested .manifest files, client connections load limiting. We are also releasing a big new feature (OPAL Scopes), that allows a single OPAL server deployment to serve separate scopes of policy and data (in essence: multi-tenancy).

We also did a bunch of cleanups to the repo (code formatting, packaging, updated dependencies, etc) as well as updated the base image to the official docker.

Feature: OPAL Scopes (experimental)

OPAL Scopes is a new experimental feature that allows a single OPAL server to track multiple git repositories,
and in essence enables true multi-tenancy by a single OPAL deployment.

OPAL Server can now track multiple "scopes" that each can have its own separate data sources
and policy sources (i.e: git repo) and each OPAL Client may subscribe to a single scope.

--

• OPAL Scopes: Infra (scope manager) (#247) authored by @orishavit
• Clone git repositories via celery (#248) authored by @orishavit
• OPAL Scope API (#249) authored by @orishavit
• OPAL scopes: Periodic updates (#250) authored by @orishavit
• OPAL Scopes: Client ID handling (#251) authored by @orishavit
• Check for opal_scope JWT claim when requesting scope policy (#252) authored by @orishavit
• Fix HTTP test with new target (#256) authored by @orishavit
• Set default server role to Primary (#257) authored by @orishavit
• Celery worker fixes (#258) authored by @orishavit
• Rename OPAL JWT claim to allowed_scopes (#260) authored by @orishavit
• OPAL Scopes: Policy update events (#261) authored by @orishavit
• OPAL Scopes: Data API (#262) authored by @orishavit
• Scoped docker example (#265) authored by @orishavit
• Opal scopes fixes (#288) authored by @asafc

Feature: Restricted topics

You can now add a permitted_topics claim on an OPAL token (when generating the token with the /token endpoint) that is enforced by the OPAL server, and restricts what topics a client can subscribe to.

--

• Added optional permitted_topics (#214) authored by @idan-ben-ami

Feature: Nested manifests

Manifests allow you to determine the order rego policies are loaded into OPA. We now support multiple nested manifest files (instead of one per repo):
Each line in the .manifest file should contain a path, relative to the .manifest file, which could be one of the following:

• A policy / data file (i.e: a .rego file or data.json file).
• A folder, containing another .manifest file to be loaded recursively.

--

• Support nested manifests and relative paths (#216) authored by @roekatz

Feature: Client load limiting

OPAL server can now (load) limit the amount of connections from OPAL clients and return 429.

Check out this docker compose example as well as the PR notes.

--

• Introduce client load limiting feature (#221) authored by @roekatz

Base image improvements

• Only install rookout deps for a dedicated image tag (#253) authored by @roekatz
• Rk/python3.10 image base (#254) authored by @roekatz

Project structure and tooling

• Refactor packages file tree (#220) authored by @orishavit
• pre-commit with black, isort, docformatter (#243) authored by @singingwolfboy
• Unpin dependencies in our various requires.txt files (#239) authored by @singingwolfboy
• Add dependency on asgiref (#263) authored by @singingwolfboy

Bug fixes and small improvements

• fix race condition in example opal docker configs (#219) authored by @asafc
• Remove un-needed usage of len() (#215) authored by @MatanyaStroh
• fix ref before assignment (#223) authored by @obsd
• fix webhook endpoint when git source is api bundle server (#226) authored by @asafc
• change ddtrace to trace all processes (#227) authored by @obsd
• Remove unused imports + fix run-with-security script (#244) authored by @asafc
• refix: remove unused imports (#245) authored by @asafc
• Return a full bundle when client requests a base revision that doesn't exist (#285) authored by @orishavit
• Log exception and retry if thrown in polling task (#287) authored by @orishavit
• Pubsub: Use new broadcaster disconnection bugfix (conditioned with a feature flag) (#246) authored by @roekatz
• Read OPAL_LOG_SERIALIZE as boolean (#280) authored by @michivi
• Add retry options for the policy store (#282) authored by @michivi
• Fix pipe opa logs (#267) authored by @asafc
• Redis broadcaster URI does not support password authentication (#279) authored by @treysidechain
• Update README.md (#237) authored by @orweis

New Contributors

• @idan-ben-ami made their first contribution in (#214)
• @MatanyaStroh made their first contribution in (#215)
• @orishavit made their first contribution in (#220)
• @singingwolfboy made their first contribution in (#239)
• @treysidechain made their first contribution in (#279)
• @michivi made their first contribution in (#280)

Full Changelog: 0.1.21...0.2.0

v0.1.21

• Added a docker compose example how to use kafka as broadcaster
• Added a doc with explanations: How to run OPAL with Kafka
• Added the ability to send messages from kafka directly to OPAL server (#213)

v0.1.20

Bug fixes in this release:

1. Official docker container will now run as non-root user (#210):
   This improves the security of our official container and was requested by several users.

2. Use fixed clone path for containerized opal server (#211):
   This solves a race condition we observed in some kubernetes environments. If the leader worker is killed - the next leader will pick a different clone directory and will reclone the git repo. Due to the time it takes to clone a git repo this creates a race with policy bundle requests that issue a git diff-tree command to a non-cloned repo and will cause exceptions. The new fixed version will not used a randomized clone directory if running inside docker (the official docker container sets this behavior with a new config var: OPAL_POLICY_REPO_REUSE_CLONE_PATH

NOTICE:
Some power users of OPAL are know to mount paths from the container to the external filesystem. For these users, please be aware that you might need to fix your mount paths from /app/… to /opal/… which is now the WORKDIR in the new official image (was / in the old image). Check out OPAL's Dockerfile to see if you are affected. Since this is an implementation detail of OPAL we do not consider this a breaking change. No public or otherwise documented APIs are changed by this release.

All praise to @roekatz for his great work on this release!

v0.1.19

Small fixes and improvements:

• Fixed #190: When OPAL is not provided a valid passphrase, JWT signer will now output an indicative error to log (#205)
• Fixed #200: Added error if the user is trying to run a multi-worker opal server without configuring a broadcaster (#204)
• Fixed #207: Support self signed certificates in bundle server and websockets (#208)
• Rename docker hub repo - all new versions of official docker images can now be found here: https://hub.docker.com/u/permitio
• Upgrade Loguru version to 0.6.0

v0.1.18

This is a minor release, but please pay attention to the release notes about JWT issuer and JWT audience!

Changes that require your attention

• Authorizon (the company behind OPAL) is rebranding as Permit.io (more announcements will follow).
• Therefore we moved the OPAL repo under our new organization in github. Changed all references to authorizon.com as a result.
• We also changed the default AUTH_JWT_ISSUER and AUTH_JWT_AUDIENCE.

If you upgrade to OPAL 0.1.18 you need to do one of the following: (either 1 or 2, it does not matter)

1. add these config vars to your setup (will override the new defaults with the old values)

OPAL_AUTH_JWT_AUDIENCEhttps://api.authorizon.com/v1/
OPAL_AUTH_JWT_ISSUER=https://authorizon.com/

2. Recreate all JWT tokens used by your OPAL setup (i.e: client token, etc) since OPAL server will now check the tokens with a new JWT issuer and JWT audience.

Few minor fixes:

1. #185: JWT signer - clearer error message for JWT signer with bad input
2. #186: added an example docker compose using secure mode with tokens. all you need is to clone the repo, cd to the docker/ directory and run ./run-example-with-security.sh. full instructions are here
3. #187: fixed healthcheck policy in policy-only mode (fixes #181)
4. #188: fix bug where changes to different directories in the same commit are ignored, full notes in the pull (fixes #180)
5. #189: add conditional APM to opal server (added option to configure opal with datadog)

v0.1.17

Two major new features in this release:

1) More data in healthcheck policy to help keep track of OPA state

#163: Updated the healthcheck policy to add more data about the current synchronization state of OPAL. The new data format is described in the original issue (#154).

2) Add broadcaster keepalive task

#177: Add broadcaster keepalive task to prevent broadcast backbone from closing the session

We have seen a behavior in our demo environment - where Postgres closes a db session used for broadcaster notifications after no messages were broadcasted for several days.

We are adding a safeguard here (a keepalive task) so that if in a live environment there is no traffic for a long period of time - the session will be kept alive due to the keepalive messages triggering a NOTIFY event, which in turn will also keep the LISTEN sessions alive due to traffic going through.

3) Added OPAL server statistics tracking about connected clients and topics

OPAL server can now keep track of connected clients and display a statistics API.
The feature was suggested in #155 and was added in #169

Check out this docker compose config on how to turn the statistics feature. The new statistics endpoint is https://opalserver.com/statistics.

---

Thanks @obsd for these excellent new features and thanks @hongbo-miao for opening the issues and helping us spec these features!

v0.1.16

This release brings a major new feature: pluggable policy sources (which addresses the feature request in #131)

• Along with the ability to source policy (rego files) from git, opal 0.1.16 adds the options to serve policy files into opal from an external API that serve tar bundles.
• the format of such files is the same as OPA bundles
• We added a document explaining how Sourcing policy from API works
• We added an example docker compose configuration with policy sourced from API instead of git.
• Major props to @obsd - our newest addition to the OPAL team - for the launch of his first feature :)

This release also adds few minor fixes and improvement:

• #161 is fixed: we added the healthcheck policy to the manifest, so OPA healthcheck policy should work in kubernetes just fine :)
• We increased the log verbosity to make it more clear how messages are routed within OPAL.

v0.1.15

This version includes minor changes to log messages:

• Updated to latest RPC and Pub/Sub library versions (added more logs)
• Added env var OPAL_LOG_FORMAT_INCLUDE_PID - if enabled - will add the process id to the logger format (helps debugging in case of multi-worker setup)
• Locked some dependency library versions, i.e: websockets and pyjwt due to breaking changes in latest versions
• fixed issue with duplicate log messages in uvicorn loggers