diff --git a/.cspell-docs.json b/.cspell-docs.json index 331647eebd..79ab867bb4 100644 --- a/.cspell-docs.json +++ b/.cspell-docs.json @@ -30,6 +30,8 @@ "./docs/docs/dev/setting_up_development.md" ], "ignoreRegExpList": [ + "ABAC", + "gcloud", "Urls", "saturnia", "Niels", diff --git a/docs/docs/dev/auth_flow.md b/docs/docs/dev/auth_flow.md index 53f0c1cbdd..bece3da6cf 100644 --- a/docs/docs/dev/auth_flow.md +++ b/docs/docs/dev/auth_flow.md @@ -2,6 +2,35 @@ Bacalhau authenticates and authorizes users in a multi-step flow. +## Requirements + +We know our potential users have many possible requirements around auth and +exist across the entire spectrum from "no auth needed because its a simple local +deployment" to "enterprise-grade security for publicly accessible nodes". Hence, +the auth system needs to be unopinionated about how authentication and +authorization gets achieved. + +The auth system has therefore been designed with a few goals in mind: + +- **Flexible authentication**: it should be easy for users to add their own + authentication method, including simple methods like using shared secrets and + more complex methods up to OAuth and OIDC. +- **Flexible authorization**: it should be possible for users to be authorized + based on a number of different modes, including group-based auth, RBAC and + ABAC. The exact permissions of each should be customizable. The system should + not require, for example, a particular model of "namespaces" or "workspaces" + because these don't necessarily fit all use cases. +- **Future proofing**: the auth system should not require core-level upgrades + to support advancements in cryptography. The hash functions and key sizes that + are considered "secure" change over time, so the Bacalhau core should not be + forced to have an opinion on this by the auth system and should not have to + play "whack-a-mole" with supporting different configurations for different + customers. Instead, it should be possible for customers to apply a policy that + makes sense for them and upgrade security at their own pace. +- **Performance**: any calls to remote servers or complex algorithms to decide + logic should happen once in the authentication process, and then subsequent + calls to the API should introduce little overhead from authorization. + ## Roles - **Auth server** is a set of API endpoints that are trusted to make auth @@ -20,13 +49,22 @@ Bacalhau implements flexible authentication and authorization using policies which are written using a machine-executable policy format called Rego. - Each **authentication policy** receives authentication credentials as input - and outputs JWT access tokens that will supplied to future API calls. + and outputs access tokens that will supplied to future API calls. - Each **authorization policy** receives access tokens as input and outputs decisions about allowable access to APIs and job submission. These two policies work together to define the entire authentication and authorization scheme. +# Auth flow + +The basic list of steps is: + +1. Get the list of acceptable authn methods +2. Pick one and execute it, collecting any credentials from the user +3. Submit the credentials to the authn API +4. Receive an access token and use it in all future requests + ## 1. Retrieve list of supported authentication methods User agents make a request to their configured auth server to retrieve a list of @@ -66,6 +104,14 @@ Each authentication method object describes: * parameters to be used in running the authentication method, specific to that type +Each "type" can be used to implement a number of different authentication +methods. The types broadly correlate with behavior that the user agent needs to +take to run the authentication flow, such that there can be a single piece of +user agent code that is capable of running each type, with different input +parameters. + +The supported types are: + ### `challenge` authentication This method is used to identify users via a private key that they hold. The @@ -116,8 +162,17 @@ submit data for a "userpass" method, the user agent would POST to ## 3. Auth server checks the authn data against a policy The auth server processes the request by inputting the auth credentials into a -auth policy. If the auth policy finds the passed data acceptable, it returns a -signed JWT that the user can use as an access token. +auth policy. If the auth policy finds the passed data acceptable, it returns an +access token that the user can use in subsequent calls. + +(Aside: there is actually no specification on the structure of the access token. +The user agent should treat it as an opaque blob that it receives from the auth +server and submits to the API server. Currently, all of the core Bacalhau code +also does not have any opinion of the auth token – it is not assumed to be any +specific type of object, and all parsing and handling is handled by the Rego +policies. However, all of the currently implemented Rego policies output and +expect JWTs, and it is recommended that users continue to use this convention. +The rest of this document will assume access tokens are JWTs.) The signed JWT is returned to the user agent. The user agent takes appropriate steps to keep the access token secret. @@ -156,17 +211,17 @@ The timestamp after which the token is no longer valid. A map of namespaces to permission bits. -The key in the map is an namespace name that the user has some level of access +The key in the map is a namespace name that the user has some level of access of. Namespace names are ephemeral – i.e. there does not need to be a persistent or coordinated store of namespaces shared across the whole cluster. Instead, the **format** of namespace names is an interface for the network operator to decide. For example, the default policy will just give the user access to a namespace -identifier by the `sub` field (e.g. their username). But in principle, more +identified by the `sub` field (e.g. their username). But in principle, more complex setups involving groups could be used. -Namespace names can contain a `*`, which by convention will match any set of +Namespace names can be a `*`, which by convention will match any set of characters, like a filesystem glob. But it is up to the various auth policies to actually implement this. So a JWT claim containing `"*"` would give default permissions for all namespaces. @@ -184,7 +239,9 @@ following bits are set: The user agent includes an `Authorization` header with the access token it wishes to use passed as a bearer token: - Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpX3459… +``` +Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpX3459… +``` Note that the `Authorization` header is strictly optional – access for unauthorized users is controlled using the policy, and may be allowed. The API @@ -201,3 +258,73 @@ node APIs, the policy may make a blanket decision simply using whether the user has an authorization token or not, or may choose to make a decision depending on the type of authorization. For namespaced APIs, such as job APIs, the policy should examine the namespaces in the JWT token and respond accordingly. + +The authz server will return a `403 Forbidden` error if the user is not allowed +to carry out the requested action. It will also return a `401 Unauthorized` +error if the token the user passed is not valid for any future request. In the +latter case, the user agent should discard the token and execute the above flow +again to get a new one. + +# Future work + +There are a number of roadmap items that will enhance the auth system: + +## Authn/z in the Web UI + +The Web UI currently does not have any authn/z capability, and so can only work +with the default Bacalhau configuration which does not limit unauthenticated +users from querying read-only API endpoints. + +To upgrade the Web UI to work in authenticated cases, it will be necessary to +implement the algorithms noted above. In short: + +1. The Web UI will need to query the auth API endpoint for available authn + methods. +2. It should then pick an appropriate authn method, either by asking the user, + choosing based on known available data (e.g. existing presence of a private + key), or by picking the only available option. +3. It should then run the authn flow for that type: + - For `challenge` types, it will need a private key. It should probably + generate and store one persistently rather than asking the user to upload + theirs. + - For `ask` types, it will need to parse the input JSON Schema and present a + web form to collect the necessary authn credentials. +4. Once it has successfully authenticated, it should persistently store the + access token and add it to all subsequent API requests. + +## Addition of an `external` authentication type + +This type will power future OAuth2/OIDC authentication. The principle is that: + +1. The type will specify a remote endpoint to redirect the user to. The CLI will + open a browser to this endpoint (or otherwise advise the user to do this) and + the Web UI will just issue a redirect to this endpoint. + +2. The user completes authentication at the remote service and is then + redirected back to a supplied endpoint with valid credentials. + + The CLI may need to run a temporary web server to receive the redirect (this + is how CLI tools like `gcloud` currently handle the OIDC flow). The Web UI + will need to specify a redirect that it can subsequently decode credentials + for. + + Also specified in the authentication method data will be any query + parameters that the CLI/WebUI needs to populate with the redirect path. E.g. + the specific OIDC scheme might specify the return location as a `?redirect` + url query parameter, and the authentication type should specify the name of + this parameter. + +3. There doesn't need to be an optional step where the user exchanges the + identity token they received from the remote auth server for a Bacalhau auth + token. Instead, the system could just use the returned credential directly. + + However, this may be a beneficial step for mapping OIDC credentials into e.g. + a JWT that specifies available namespaces. So there should probably be a step + where the token received from the OIDC flow is passed to the authn method + endpoint, and a policy has the chance to return a different token. In the + basic case, it can check the validity of the token and return it unchanged. + +4. The returned credential will be a JWT or similar access token. The user agent + should use this credential to query the API as above. The authz policy should + be configured to recognize these access tokens and apply authz control based + on their content, as for the other methods. diff --git a/docs/docs/setting-up/running-node/auth.md b/docs/docs/setting-up/running-node/auth.md index 02e27fdf46..e8d031bc13 100644 --- a/docs/docs/setting-up/running-node/auth.md +++ b/docs/docs/setting-up/running-node/auth.md @@ -1,7 +1,7 @@ --- sidebar_label: 'Authentication and authorization' sidebar_position: 160 -description: How to configure authentication and authorization on your bacalhau node. +description: How to configure authentication and authorization on your Bacalhau node. --- # Authentication and authorization @@ -17,10 +17,10 @@ system. "Anonymous mode" is only appropriate for testing or evaluation setups. In anonymous mode, Bacalhau will allow: -- Users identified by a self-generated private key to submit any job, cancel - their own jobs, read job lists and describing jobs. +- Users identified by a self-generated private key to submit any job and cancel + their own jobs. - Users not identified by any key to access other read-only endpoints, such as - node or agent information. + to read job lists, describe jobs, and query node or agent information. ## Restricting anonymous access @@ -31,8 +31,10 @@ Restricting API access to only users that have authenticated requires specifying a new **authorization policy**. You can download a policy that restricts anonymous access and install it by using: - curl -sL https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authz/policies/policy_ns_anon.rego -o ~/.bacalhau/no-anon.rego - bacalhau config set Auth.AccessPolicyPath ~/.bacalhau/no-anon.rego +``` +curl -sL https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authz/policies/policy_ns_anon.rego -o ~/.bacalhau/no-anon.rego +bacalhau config set Auth.AccessPolicyPath ~/.bacalhau/no-anon.rego +``` Once the node is restarted, accessing the node APIs will require the user to be authenticated, but by default will still allow users with a self-generated key @@ -43,15 +45,15 @@ specifying a new **authentication policy**. You can download a policy that restricts key-based access and install it by using: ``` - curl -sL https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authn/challenge/challenge_ns_no_anon.rego -o ~/.bacalhau/challenge_ns_no_anon.rego - bacalhau config set Auth.Methods '\{Method: ClientKey, Policy: \{Type: challenge, PolicyPath: ~/.bacalhau/challenge_ns_no_anon.rego\}\}' +curl -sL https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authn/challenge/challenge_ns_no_anon.rego -o ~/.bacalhau/challenge_ns_no_anon.rego +bacalhau config set Auth.Methods '\{Method: ClientKey, Policy: \{Type: challenge, PolicyPath: ~/.bacalhau/challenge_ns_no_anon.rego\}\}' ``` Then, modify the `allowed_clients` variable in `challange_ns_no_anon.rego` to include acceptable client IDs, found by running `bacalhau id`. ``` - bacalhau id | jq -rc .ClientID +bacalhau id | jq -rc .ClientID ``` Once the node is restarted, only keys in the allowed list will be able to access @@ -63,14 +65,159 @@ Users can authenticate using a username and password instead of specifying a private key for access. Again, this requires installation of an appropriate policy on the server. - curl -sL https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authn/ask/ask_ns_password.rego -o ~/.bacalhau/ask_ns_password.rego - bacalhau config set Auth.Methods '\{Method: Password, Policy: \{Type: ask, PolicyPath: ~/.bacalhau/ask_ns_password.rego\}\}' +``` +curl -sL https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authn/ask/ask_ns_password.rego -o ~/.bacalhau/ask_ns_password.rego +bacalhau config set Auth.Methods '\{Method: Password, Policy: \{Type: ask, PolicyPath: ~/.bacalhau/ask_ns_password.rego\}\}' +``` Passwords are not stored in plaintext and are salted. The downloaded policy expects password hashes and salts generated by `scrypt`. To generate a salted password, the helper script in `pkg/authn/ask/gen_password` can be used: - cd pkg/authn/ask/gen_password && go run . +``` +cd pkg/authn/ask/gen_password && go run . +``` This will ask for a password and generate a salt and hash to authenticate with it. Add the encoded username, salt and hash into the `ask_ns_password.rego`. + +# Writing custom policies + +In principle, Bacalhau can implement any auth scheme that can be described in a +structured way by a policy file. + +Policies are written in a language called +[Rego](https://www.openpolicyagent.org/docs/latest/policy-language/), also used +by Kubernetes. Users who want to write their own policies should get familiar +with the Rego language. + +## Custom authentication policies + +Bacalhau will pass information pertinent to the current request into every +authentication policy query as a field on the `input` variable. The exact +information depends on the type of authentication used. + +### `challenge` authentication + +`challenge` authentication uses identifies the user by the presence of a private +key. The user is asked to sign an input phrase to prove they have the key they +are identifying with. + +Policies used for `challenge` authentication do not need to actually implement +the challenge verification logic as this is handled by the core code. Instead, +they will only be invoked if this verification passes. + +Policies for this type will need to implement these rules: + +* `bacalhau.authn.token`: if the user should be authenticated, an access token + they should use in subsequent requests. If the user should not be + authenticated, should be undefined. + +They should expect as fields on the `input` variable: + +* `clientId`: an ID derived from the user's private key that identifies them + uniquely +* `nodeId`: the ID of the requester node that this user is authenticating with +* `signingKey`: the private key (as a JWK) that should be used to sign any + access tokens to be returned + +The simplest possible policy might therefore be this policy that returns the +same opaque token for all users: + +```rego +package bacalhau.authn + +token := "anything" +``` + +A more realistic example that returns a signed JWT is in +[challenge_ns_anon.rego](https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authn/challenge/challenge_ns_no_anon.rego). + +### `ask` authentication + +`ask` authentication uses credentials supplied manually by the user as +identification. For example, an `ask` policy could require a username and +password as input and check these against a known list. `ask` policies do all +the verification of the supplied credentials. + +Policies for this type will need to implement these rules: + +* `bacalhau.authn.token`: if the user should be authenticated, an access token + they should use in subsequent requests. If the user should not be + authenticated, should be undefined. +* `bacalhau.authn.schema`: a static JSON schema that should be used to collect + information about the user. The `type` of declared fields may be used to pick + the input method, and if a field is marked as `writeOnly` then it will be + collected in a secure way (e.g. not shown on screen). The `schema` rule does + not receive any `input` data. + +They should expect as fields on the `input` variable: + +* `ask`: a map of field names from the JSON schema to strings supplied by the + user. The policy should validate these credentials. +* `nodeId`: the ID of the requester node that this user is authenticating with +* `signingKey`: the private key (as a JWK) that should be used to sign any + access tokens to be returned + +The simplest possible policy might therefore be one that asks for no data and +returns the same opaque token for every user: + +``` +package bacalhau.authn + +schema := {} +token := "anything" +``` + +A more realistic example that returns a signed JWT is in +[ask_ns_example.rego](https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authn/ask/ask_ns_example.rego). + +## Custom authorization policies + +Authorization policies do not vary depending on the type of authentication used +– Bacalhau uses one authz policy for all API requests. + +Authz policies are invoked for every API request. Authz policies should check +the validity of any supplied access tokens and issue an authz decision for the +requested API endpoint. It is not required that authz policies enforce that an +access token is present – they may choose to grant access to unauthorized users. + +Policies will need to implement these rules: + +* `bacalhau.authz.token_valid`: true if the access token in the request is + "valid" (but does not necessarily grant access for this request), or false if + it is invalid for every request (e.g. because it has expired) and should be + discarded. +* `bacalhau.authz.allow`: true if the user should be permitted to carry out the + input request, false otherwise. + +They should expect as fields on the `input` variable for both rules: + +* `http`: details of the user's HTTP request: + * `host`: the hostname used in the HTTP request + * `method`: the HTTP method (e.g. `GET`, `POST`) + * `path`: the path requested, as an array of path components without slashes + * `query`: a map of URL query parameters to their values + * `headers`: a map of HTTP header names to arrays representing their values + * `body`: a blob of any content submitted as the body +* `constraints`: details about the receiving node that should be used to validate any supplied tokens: + * `cert`: keys that the input token should have been signed with + * `iss`: the name of a node that this node will recognize as the issuer of any signed tokens + * `aud`: the name of this node that is receiving the request + +Notably, the `constraints` data is appropriate to be passed directly to the Rego +`io.jwt.decode_verify` method which will validate the access token as a JWT +against the given constraints. + +The simplest possible authz policy might be this one that allows all users to +access all endpoints: + +```rego +package bacalhau.authz + +allow := true +token_valid := true +``` + +A more realistic example (which is the Bacalhau "anonymous mode" default) is in +[policy_ns_anon.rego](https://raw.githubusercontent.com/bacalhau-project/bacalhau/main/pkg/authz/policies/policy_ns_anon.rego).