Policy Versioning

[renuka-fernando]

1. Policy Versioning in Policy Engine

Policy Engine can be built using one or more versions of the same policy.
For example the following both policies can be live at the same time.

• ModifyHeaders v1.0.0
• ModifyHeaders v1.0.1

1.1. versionResolution

Alternative names for versionResolution:

• versionConstraint
• updateStrategy

Available versionResolution:

1. exact -> Pin to exact version, no updates (v1.0.0 → v1.0.0)
2. patch -> Allow patch updates only (v1.0.0 → latest v1.0.x)
3. minor -> Allow minor and patch updates (v1.0.0 → latest v1.x.x) [DEFAULT]

Note: major is not available as it is dangerous.
Default to minor if not specified.

1.2. Policy Manifest File - policy-manifest.yaml (Used in build time)

1.2.1. User Provided File

The same policy can be defined multiple times with different versioning strategies. Because different APIs or the Same API may require different versions of the same policy.

User can define the versionResolution at the root level and override it at the individual policy level.

version: v1/alpha1
versionResolution: minor
policies:
  - name: BasicAuth
    version: v1.0.0
    versionResolution: exact # Override the root level versionResolution for this policy.
  - name: BasicAuth
    version: v1.0.1
    versionResolution: exact # Same BasicAuth policy with Exact versioning
  - name: BasicAuth
    version: v1.0.2
    versionResolution: minor # Same BasicAuth policy with Minor versioning
  - name: MyCustomPolicy
    version: v1.0.0
    versionResolution: exact # User can define this but this is N/A for local policies
    filePath: ./my-custom-policy/v1.0.0

1.2.2. Policy Manifest Lock file - policy-manifest-lock.yaml

This contains the exact versions resolved based on the versionResolution defined in the policy-manifest.yaml file. This file also included in the gateway image to ensure the same versions are used when the gateway is deployed. User can even commit this file in the VCS to have a record of the exact versions used in the build.

In the following example, gateway load all the versions of the BasicAuth policy.

version: v1/alpha1
policies:
  - name: BasicAuth
    version: v1.0.0
    checksum: sha256:abc123...
    source: hub
  - name: BasicAuth
    version: v1.0.1
    checksum: sha256:abc123...
    source: hub
  - name: BasicAuth
    version: v1.0.8 # Resolved version based on the update strategy defined in policy-manifest.yaml
    checksum: sha256:abc123...
    source: hub
  - name: MyCustomPolicy
    version: v1.0.0
    checksum: sha256:abc123...
    source: local
    filePath: ./my-custom-policy/v1.0.0

2. Policy Versioning in API YAML (Used in runtime when APIs are deployed to the Gateway)

Same versionResolution concepts applied here. Default to minor if not specified. The versionResolution resolved to the latest in the gateway not in the Policy Hub.

apiVersion: gateway.api-platform.wso2.com/v1alpha1
kind: RestApi
metadata:
  name: weather-api-v1.0
spec:
  name: Weather-API
  version: v1.0
  context: /weather/$version
  upstream:
    main:
      url: http://sample-backend:5000/api/v2
  operations:
    - method: GET
      path: /{country_code}/{city}
      policies:
        - name: ModifyHeaders
          version: v1.0.0
          versionResolution: exact # Get exact version 1.0.0
          params:
            requestHeaders:
              - action: SET
                name: operation-level-req-header
                value: hello
            responseHeaders:
              - action: SET
                name: operation-level-res-header
                value: world
    - method: GET
      path: /alerts/active
    - method: POST
      path: /alerts/active
      policies:
        - name: Respond
          version: v1.0.0
          versionResolution: minor # Get latest minor version in the gateway (not in the Policy Hub)
          params:
            statusCode: 201
            body: '{"message": "Active weather alert created successfully."}'
            headers:
              - name: Content-Type
                value: application/json

2.1. Version Resolution in the Policy Engine

Build-time: Gateway image contains (from lock file)

  policies:
    - BasicAuth v1.0.0
    - BasicAuth v1.0.1
    - BasicAuth v1.0.8

Runtime: API deployed asking for

  operations:
    - policies:
      - name: BasicAuth
        version: v1.0.5
        versionResolution: minor

Policy Engine will pick BasicAuth v1.0.8 as it is the latest minor version (actually the latest patch) available in the gateway image.

3. Version Resolution Workflow

1. User defines the policies in the policy-manifest.yaml with version and versionResolution.
2. User runs the following command to build the gateway.

   apipctl gateway build -f policy-manifest.yaml \
      --docker-registry ghcr.io/renuka-fernando/my-gateway \
      --image-tag v1.0.2 \
      [--push] \
      [--no-cache] \
      [--platform linux/amd64] \
      [--policy-hub https://policy-hub.wso2.com]
   

3. apipctl sends the policy-manifest.yaml to the Policy Hub to resolve the versions based on the versionResolution defined.
4. Policy Hub skips local policies and resolves the versions for the rest of the policies based on the versionResolution defined.
5. Policy Hub returns the resolved versions to apipctl.
6. apipctl generates the policy-manifest-lock.yaml file.
7. apipctl builds the gateway image with the resolved policies.

[nuwand]

Expecting the api developer to specify the version resolution strategy seems too much. Can't we just leave it for the Gateway admin only?

i.e. API developer always works with major.minor only. Gateway admin decides which version to bound to based on the policies.yaml.

[renuka-fernando]

+1

With the offline discussion, we decided to go with just the major version in the API YAML. So the API developer just provides the major version.

[renuka-fernando]

Included downloadURL as well in the policy-manifest-lock.yaml. apipctl will use this link to download each policy.

[renuka-fernando]

We don't need this downloadURL in the lock file. It is not required to write the whole payload got from the Policy Hub to this lock file.

[renuka-fernando]

Downloading policies from Policy Hub

Option 1: Using apipctl (Recommended)

The apipctl CLI tool reads the policy-manifest.yaml file, sends the JSON request to the Policy Hub to resolve the versions based on the versionResolution defined, downloads the policies, and generates the policy-manifest-lock.yaml file with the resolved versions, provide the policy-manifest-lock.yaml to the gateway builder to build gateway images. In this approach, the policy file path should be communicated to the gateway builder.

In this case, policy-manifest-lock.yaml will look like this:

The filePath is included in the lock file to provide the gateway builderwith the location of the downloaded policies.

version: v1/alpha1
policies:
  - name: BasicAuth
    version: v1.0.0
    checksum: sha256:abc123...
    source: hub
    filePath: ./basic-auth/v1.0.0 # Include filePath
  - name: BasicAuth
    version: v1.0.1
    checksum: sha256:abc123...
    source: hub
    filePath: ./basic-auth/v1.0.1
  - name: BasicAuth
    version: v1.0.8
    checksum: sha256:abc123...
    source: hub
    filePath: ./basic-auth/v1.0.8
  - name: MyCustomPolicy
    version: v1.0.0
    checksum: sha256:abc123...
    source: local
    filePath: ./my-custom-policy/v1.0.0

Option 2: Gateway Builder

The gateway builder can directly communicate with the Policy Hub to resolve the versions and download the policies. In this approach, the gateway builder should read the policy-manifest.yaml file, send the JSON request to the Policy Hub to resolve the versions based on the versionResolution defined, download the policies, and generate the policy-manifest-lock.yaml file with the resolved versions, and build the gateway image.

[renuka-fernando]

We agreed on going with option 1, but excluding the filepath for policy hub policies. We are going to follow a file path structure.

[renuka-fernando]

So the updated lock file.

version: v1/alpha1
policies:
  - name: BasicAuth
    version: v1.0.0
    checksum: sha256:abc123...
    source: hub
  - name: BasicAuth
    version: v1.0.1
    checksum: sha256:abc123...
    source: hub
  - name: BasicAuth
    version: v1.0.8
    checksum: sha256:abc123...
    source: hub
  - name: MyCustomPolicy
    version: v1.0.0
    checksum: sha256:abc123...
    source: local

The file path format is defined by use that is <policy-name-in-kebab-case>-v<version>

[renuka-fernando]

We are going to use go based new approach to handle dependencies.
Discussion: #747

With this method, only one major version of policy can be installed on the Policy Engine.
Eg: AddHeader v1.3.5, AddHeader v2.2.3 (AddHeader v2.4.5 can't be supported - only one major version is supported)

The API YAML definition only specifies the major version of the policy. The policy engine applied the relevant version installed on it.

apiVersion: gateway.api-platform.wso2.com/v1alpha1
kind: RestApi
metadata:
  name: weather-api-v1.0
spec:
  name: Weather-API
  version: v1.0
  context: /weather/$version
  upstream:
    main:
      url: http://sample-backend:5000/api/v2
  operations:
    - method: GET
      path: /{country_code}/{city}
      policies:
        - name: ModifyHeaders
          version: v1
          params:
            requestHeaders:
              - action: SET
                name: operation-level-req-header
                value: hello
            responseHeaders:
              - action: SET
                name: operation-level-res-header
                value: world
    - method: GET
      path: /alerts/active
    - method: POST
      path: /alerts/active
      policies:
        - name: Respond
          version: v2
          params:
            statusCode: 201
            body: '{"message": "Active weather alert created successfully."}'
            headers:
              - name: Content-Type
                value: application/json

[renuka-fernando]

cc: @nuwand @malinthaprasan @pubudu538

[nuwand]

Since the API Developer specifies only the major version of the policy (on the api.yaml) API they would expect the same version of the policy to be executed everywhere it is mentioned right? i.e. They would expect modify_header v2 to be the same on all resources. So this should be fine AFAIU.

[DakshithaS]

So, in this approach, minor versions must be treated as behavior-stable from an API consumer’s point of view.
This means we cannot introduce behavioral changes such as:

• Changing default values
• Changing policy execution order
• Changing how errors are handled

And if we do need to make these kinds of changes, do we always have to introduce a new major version of the policy?

[renuka-fernando]

Since the API Developer specifies only the major version of the policy (on the api.yaml) API they would expect the same version of the policy to be executed everywhere it is mentioned right? i.e. They would expect modify_header v2 to be the same on all resources. So this should be fine AFAIU.

Yes, it is even possible to mix v1 and v2 in the same API.

apiVersion: gateway.api-platform.wso2.com/v1alpha1
kind: RestApi
metadata:
  name: weather-api-v1.0
spec:
  name: Weather-API
  version: v1.0
  context: /weather/$version
  upstream:
    main:
      url: http://sample-backend:5000/api/v2
  operations:
    - method: GET
      path: /{country_code}/{city}
      policies:
        - name: ModifyHeaders
          version: v1
          params:
            requestHeaders:
              - action: SET
                name: operation-level-req-header
                value: hello
        - name: ModifyHeaders
          version: v2
          params:
            requestHeaders:
              - action: SET
                name: operation-level-req-header
                value: hello

So, in this approach, minor versions must be treated as behavior-stable from an API consumer’s point of view. This means we cannot introduce behavioral changes such as:

• Changing default values
• Changing policy execution order
• Changing how errors are handled

And if we do need to make these kinds of changes, do we always have to introduce a new major version of the policy?

Exactly.

i.e., if the admin rebuilds the Gateway with a new ModifyHeaders policy (from v2.3.1 to v2.5.6) that affects all the APIs using the ModifyHeaders v2 policy. v2.3 and v2.5 should be compatible because in API level, minor versions can't be defined.