API Key Authentication policy

[thivindu]

Problem

APIs need a standard, attachable policy to secure them using API keys.

Proposed Solution

Introduce an "API Key Authentication" policy that can be attached to APIs to secure them using API keys. The policy declares how the API key is presented and how to process it in the policy engine for validation, and enables the gateway to validate keys by calling the appropriate backend that issued the key.

Policy parameters:

• in: one of "header" or "query" — where the API key is expected to appear in requests.
• key: the header name or query parameter name where the API key is provided.
• value-prefix (optional): an optional prefix that should be stripped from the API key value prior to validation (for example "Bearer ").

API keys will be opaque tokens rather than JWTs. This is because opaque tokens simplifies revocation and lifecycle management.

Key lifecycle and validation:

• API keys may be generated by the gateway, the management portal, or the developer portal.
• Requests with API keys will hit the gateway; therefore validation will happen at the gateway layer to secure the API.
• The gateway should apply the policy's "in", "key", and "value-prefix" rules to extract and normalize the key prior to calling the validation service.
• The policy should return meaningful failure reasons (missing key, invalid key, expired/revoked key) so the gateway can produce appropriate responses and logs.
• Use caching in the gateway policy engine to reduce service calls.

[thivindu]

API Key Generation and Validation

API keys can be generated by the gateway, the management portal, or the developer portal. Each issuing component stores generated API keys in its own database.

Since the API key itself carries no issuer information, the gateway will need a way to determine which component issued a given key in order to validate that key. Possible approaches include:

1. Push all the API keys generated by any service to the gateway controller - Policy engine calls the gateway controller to validate the API key.
   • Pros
     • Validation is easy no service calls to outside the gateway.
     • Token revocation is simplified - pass the token revocation calls to the gateway controller too.
   • Cons
     • Each management portal or dev portal requires having a connection to the gateways to push the keys - couples the portals and the gateways.
2. Include the possible issuers as a policy parameter that the user can set when attaching a policy to an API - The gateway controller will call the service that the API key was generated and validate the API key.
   • Pros
     • Reduced service calls and simplified logic to validate a API key.
   • Cons
     • Too much to ask from the API developer since they may not be aware where API keys for the API may be generated.
     • Each gateway needs to be aware of the management portal or dev portal and required making a connection to validate the API keys - couples the gateways and the portals.
3. Use a short prefix in API key that indicates issuer - The gateway controller will decide which service (Gateway, Management portal or Dev portal) generated the API key by looking at the API key itself and call that service to validate the API key.
   • Pros
     • Reduced service calls and to validate a API key.
   • Cons
     - Each gateway needs to be aware of the management portal or dev portal and required making a connection to validate the API keys - couples the gateways and the portals.
     - Security implications - Can tell which service generated the API key by looking at the API key itself.

In my opinion using a short prefix in API key that indicates issuer is the better approach. WDYT?

And please provide alternative approaches to these.

[thivindu]

API Key caching

Caching the API keys in the gateway policy engine (in memory) will reduce latency.

Handling revocation of API keys

Invalidate the cache if an API key is revoked.

• With approach 1 caching the API keys and invalidating the cache upon revocation is simple.
• With approaches 2 and 3 the revocation events need to sent to the gateway to invalidate the cache.

[nuwand]

I see that we are considering making the API controller (control plane of a given gateway) playing a role in validating API keys. I'm not sure that is right. So far we've only considered the router and policy engine to be involved with processing API calls. The Gateway Controller is intended as a control plane for the gateway only.

[thivindu]

The alternative would be similar to approach 1 where, push all the API keys generated by any service to the policy engine itself. The policy engine would have to store the API keys in an in memory map and validate the requests.

This has the same cons denoted in approach 1 and furthermore, we have to populate the in memory map in the case of a policy engine restart.

[nuwand]

What if we use the Gateway Controller's DB as the persistence storage for API Keys and an in-memory map in the API Key policy for runtime validations? This way,

1. When an API key is generated from the Gateway -> GW Controller puts it into the local DB and emits an event which contains the new API key information.
2. When an API key is generated from the management plane or dev portal -> The relevant GW controller is notified, which then put it into its local DB and emits an event with the new API key information.
3. When the Policy Engine restarts -> The init function in the API key policy calls the GW controller to fetch all API keys and load it into its memory.
4. All revocation functions follow the same approach as in (1) and (2) above.

[nuwand]

I'd like to see more information on this API key. Can you please provide the following?

1. The API call to obtain an API key.
2. The response of the API call.
3. The API call to revoke an API key.
4. Properties of the API key. Ex: Key name, key, expiry date, etc.
5. What happens if I forget the value of the key and want to renew it.

[thivindu]

Hi all,

All the API Keys generated from every service would have the prefix apip_.
The Service (Gateway, Management Portal, Dev portal) would push the generated API key to the gateway controller, which in turn pushes the API key to the Policy engine in memory map.

API key validation would happen against the in memory map of API keys in the policy engine.

1. The API call to obtain an API key.

POST /apis/{id}/api-key

id - Handle of the API that the API key needs to be generated for

Request body -

name:
    description: Name of the API key
operations:
    description: List of API operations the key will have access to
expires_in:
    description: Expiration duration for the API key
    properties:
        unit:
              description: Time unit for expiration
              enum:
                - seconds
                - minutes
                - hours
                - days
                - weeks
                - months
        duration:
              description: Duration value for expiration
expires_at:
    description: Expiration timestamp of the API key. Default is 90 days from creation if not specified.

All of the fields in the request body are optional.

2. The response of the API call.

Response -

status:
    description: API key generation was successful or not
message:
    description: API key generation was status message
    example: API key generated successfully
api_key:
    name:
          description: Name of the API key
    api_key:
          description: Generated API key with apip_ prefix
    apiId:
          description: Unique public identifier of the API that the key is associated with (handle of the API)
    operations:
          description: List of API operations the key will have access to
    status:
          description: Status of the API key
          enum:
            - active
            - revoked
            - expired
    created_at:
          description: Timestamp when the API key was generated
    created_by:
          description: Identifier of the user who generated the API key
    expires_at:
          description: Expiration timestamp of the API key. Default is 90 days from creation if not specified.

The api_key field in the response would be empty if the API key generation is unsuccessful.

The following are the defaults if the values are not provided in the API key generation request -

• name -> ${handle}-key-${random_string}
• operations -> ["*"] - The API key is valid for all operations in the API
• Status -> active
• created_at -> Timestamp of now
• created_by -> "system" - If the user cannot be extracted from the authentication middleware
• Default expiry is 90 days from creation if not specified.

4. Properties of the API key

In the DB the API key has the following properties -

• id -> Generated unique identifier of the API key
• name -> Name of the API key
• api_key -> API key with apip_ prefix
• handle -> Unique public identifier of the API that the key is associated with
• operations -> List of API operations the key will have access to
• status -> Status of the API key
• created_at -> Timestamp when the API key was generated
• created_by -> Identifier of the user who generated the API key
• updated_at -> Timestamp when the API key was last updated
• expires_at -> Expiration timestamp of the API key

5. What happens if I forget the value of the key and want to renew it

If an API consumer has forgot the value of the API key,

they can invoke the POST /apis/{id}/api-key with the name of the previously generated API key and the other necessary parameters and retrieve a new API key of the same name with updated expiry times.

[nuwand]

Several suggestions I think are important to consider.

1. I think default expiry should be infinite. Many people use API keys for automations and hence it becomes a hassle to rotate.
2. I suggest we bind an API key to the whole API. i.e. An API key obtained for a given API can invoke all its operations. Why? Operations are messy. For example, the definition of an operation is very different across REST, SOAP, MCP, GraphQL and so on.
3. The data model doesn't show which API an API key is bound to. When someone gets information of an API key, it needs to show which API it is associated with.

I have several questions.

1. What are the acceptable values in the status column?
2. The API call to generate a new key is a POST. What is the POST body like?
3. The API calls to get a new key and rotate/update an existing key looks the same POST /apis/{id}/api-key. Shouldn't the rotate/update call be a PUT instead? Because what we are actually doing is updating some fields of an existing key.

[thivindu]

1. The only reason behind the default expiry is security. If this is not an issue and since we provide a revocation endpoint too, we can make the expiry be infinite.
2. By default, the API key binds to the whole API. The API key generator can specify which operations that this key will provide access to if they want.
3. The data model shows the handle of the API that the API key is bound to. And we currently do not have an endpoint to retrieve the details of a API key.

Answers -

1. active, revoked and expired
2. The schema of the request body is given in my previous comment.
   Example :-

{
    "name": "test-key-1",
    "expires_in": {
        "duration": 1,
        "unit": "hours"
    },
    "operations": [
        {
            "method": "GET",
            "path": "/{country_code}/{city}"
        },
        {
            "method": "POST",
            "path": "/alerts/active"
        }
    ]
}

3. We do not have a separate endpoint to rotate an API key. An API consumer uses the same POST /apis/{id}/api-key endpoint that they used to generate the token with the name of the API key in the request body to rotate that API key. Having an extra resource to rotate the token would be unnecessary in my opinion. But having a separate PUT /apis/{id}/api-key would improve clarity.

[thivindu]

The default expiry was set to be infinite if both expires_in and expires_at are not provided in the API key generation request.

[nuwand]

1. Let's go with infinite as the default timeout.
2. I think we will require an API to get details of an API Key. Without that, one cannot know the details such as expiry time and all.

[thivindu]

Revoke API Key

Revokes a given API key of an API

DELETE /apis/{id}/api-key/{apiKey}

Request -
id -> The unique public identifier of the API of the API key that needs to be revoked
apiKey -> The API key to revoke

Response -
204 -> API key revoked successfully

Remove the API key from the gateway controller DB and the gateway controller in memory cache. Then remove the api key from the policy engine.

First validate the API exists and fetch the API key from the in memory store of the gateway controller.
If the API key does not exist in the in memory store, try to delete it from the gateway controller DB and policy engine
If the API key exists,

1. Check if the API key belongs to the specified API
2. Check if the user revoking the key is the same as the one who created it
3. Check if the API key is already revoked

If these pass, revoke the API key.

1. Update the API key in the gateway controller DB and set status to be revoked. If this fails return a failure message "failed to revoke API key"
2. Remove the API key from memory store in the gateway controller. If this fails rollback the database update and set the status of the key to be active and database return a failure message "failed to revoke API key".
3. Completely remove the API key from database. This two step process ensures that no revoked tokens are present in the policy engine.
4. Remove the api key from the policy engine. If this fails return a failure message "failed to revoke API key".

Upon the failure message "failed to revoke API key" the user might try to again revoke the API key. The aforementioned process is idempotent so that a partial removal of an API key in the case of a failure does not make the state of the different components to be inconsistent and does not fail when trying to revoke it again.

[thivindu]

The OpenAPI definition and endpoints for the API key related operations are refactored to better align with REST API practices and clarity.

Generate API key

POST /apis/{id}/api-key -> POST /apis/{id}/generate-api-key

The APIKeyGenerationRequest schema allows both expires_in (duration) and expires_at (absolute timestamp), but neither is required and there's no indication of precedence if both are provided. This ambiguity could confuse API consumers. Give precedence for the expires_at if both are provided.

List API keys

GET /apis/{id}/api-key -> GET /apis/{id}/api-keys
Improve grammar in operation description.
Retrieve the list of all the active API keys for the specified API by the user. -> Retrieve all active API keys for the specified API created by the user.

Rotate API key

PUT /apis/{id}/api-key/{apiKeyName} -> POST /apis/{id}/api-keys/{apiKeyName}/regenerate

Revoke API key

DELETE /apis/{id}/api-key/{apiKey} -> POST /apis/{id}/revoke-api-key
Do not pass the actual API Key in the URL. Specify a new RevokeRequest and specify the key in the body.

[thivindu]

API Key Hashing

The API keys need to hashed and stored to prevent exposing the api keys in an event where the database is compromised.

The random part of the API keys is 32 bytes long. The resultant random byte string of 32 bytes are then hex encoded, giving a api key length of 64 characters.
This is hashed and stored in the database and incoming API keys are validated against the stored hash.
The ID of the generated API key is hex encoded and appended to the generated random key separated by "."
Format of the API Key returned to the API consumer - apip_{hex_encoded_key}.{hex_encoded_id}

Since the generated API keys are high entropy, we use SHA256 with a random salt of 32 bytes as the hashing algorithm as default.

bcrypt and argon2id are supported strong hashing algorithms to hash API keys. Hashing the API keys using these algorithms results in a hash that is much harder to break and is Post Quantum safe. Hashing keys using these algorithms is resource intensive and the resource allocations for the gateway controller and policy engine containers from the docker compose need to be increased.

Disabling API key hashing in production environments is highly discouraged.