Skip to content

Commit a80cffd

Browse files
danyi1212OPAL Test
danyi1212
and
OPAL Test
authored
Update app-tests to use local Gitea server instead of Github (#808)
* Update app-tests to use local Gitea server instead of Github - Added a minimal Gitea service to the Docker Compose configuration for local testing. - Updated the `run.sh` script to initialize Gitea and create a test repository with policy files. - Modified the README to reflect the new testing approach and prerequisites. - Introduced new policy files and data structures for comprehensive end-to-end testing. - Improved error handling and logging in the testing scripts for better troubleshooting. * Rename test-docker job to e2e-tests in GitHub Actions workflow and simplify run script execution by removing unnecessary SSH key setup steps. * Refactor GitHub Actions workflow to improve script execution format in app-tests job * Revert * Revert --------- Co-authored-by: OPAL Test <[email protected]>
1 parent f646439 commit a80cffd

File tree

10 files changed

+524
-71
lines changed

10 files changed

+524
-71
lines changed

‎.github/workflows/tests.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,10 @@ jobs:
5252
run: |
5353
pytest
5454
55-
test-docker:
55+
e2e-tests:
5656
runs-on: ubuntu-latest
5757
timeout-minutes: 60
58+
name: E2E Tests
5859
steps:
5960
# BUILD PHASE
6061
- name: Checkout

‎app-tests/README.md

Lines changed: 44 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,62 @@
1-
# OPAL Application Tests
1+
# OPAL E2E Tests
22

3-
To fully test OPAL's core features as part of our CI flow,
4-
We're using a bash script and a docker-compose configuration that enables most of OPAL's important features.
3+
This directory contains end-to-end tests for OPAL using a local Gitea git server instead of relying on external services like GitHub.
54

6-
## How To Run Locally
5+
## Overview
76

8-
### Controlling the image tag
7+
The tests spin up a complete OPAL environment including:
8+
- A local Gitea git server to host the policy repository
9+
- Multiple OPAL server instances (2 replicas)
10+
- Multiple OPAL client instances (2 replicas)
11+
- A PostgreSQL database for broadcast communication
12+
- Policy files loaded from `opal-tests-policy-repo-main/`
913

10-
By default, tests would run with the `latest` image tag (for both server & client).
14+
## Prerequisites
1115

12-
To configure another specific version:
16+
- Docker and Docker Compose
17+
- Bash shell
18+
- Basic Unix tools (curl, git, openssl, ssh-keygen)
1319

14-
```bash
15-
export OPAL_IMAGE_TAG=0.7.1
16-
```
20+
## Running the Tests
1721

18-
Or if you want to test locally built images
22+
Simply run:
1923
```bash
20-
make docker-build-next
21-
export OPAL_IMAGE_TAG=next
24+
./run.sh
2225
```
2326

24-
### Using a policy repo
27+
The script will:
28+
1. Generate authentication keys and tokens
29+
2. Start a local Gitea server
30+
3. Create a test repository with initial policy files
31+
4. Start OPAL servers and clients
32+
5. Run various tests including:
33+
- Policy updates via git push
34+
- Data updates via API
35+
- Statistics verification
36+
- Broadcast channel disconnection handling
2537

26-
To test opal's git tracking capabilities, `run.sh` uses a dedicated GitHub repo ([opal-tests-policy-repo](https://github.com/permitio/opal-tests-policy-repo)) in which it creates branches and pushes new commits.
38+
## Test Policy Files
2739

28-
If you're not accessible to that repo (not in `Permit.io`), Please fork our public [opal-example-policy-repo](https://github.com/permitio/opal-example-policy-repo), and override the repo URL to be used:
29-
```bash
30-
export [email protected]:your-org/your-repo.git
31-
```
32-
33-
As `run.sh` requires push permissions, and as `opal-server` itself might need to authenticate GitHub (if your repo is private). If your GitHub ssh private key is not stored at `~/.ssh/id_rsa`, provide it using:
34-
```bash
35-
# Use an absolute path
36-
export OPAL_POLICY_REPO_SSH_KEY_PATH=$(realpath ./your_github_ssh_private_key)
37-
```
40+
The test policies are stored in `opal-tests-policy-repo-main/` and include:
41+
- `rbac.rego` - Role-based access control policies
42+
- `utils.rego` - Utility functions
43+
- `policy.cedar` - Cedar policy examples
44+
- `data.json` - Initial data
45+
- `.manifest` - Repository manifest
3846

47+
## Troubleshooting
3948

40-
### Putting it all together
49+
If tests fail:
50+
1. Check Docker logs: `docker compose -f docker-compose-app-tests.yml logs`
51+
2. Ensure ports 3000, 7002-7003, 7766-7767, 8181-8182 are available
52+
3. Clean up and retry: `docker compose -f docker-compose-app-tests.yml down -v`
4153

42-
```bash
43-
make docker-build-next # To locally build opal images
44-
export OPAL_IMAGE_TAG=next # Otherwise would default to "latest"
54+
The script automatically retries up to 5 times to handle transient failures.
4555

46-
export [email protected]:your-org/your-repo.git # To use your own repo for testing (if you're not an Permit.io employee yet...)
47-
export OPAL_POLICY_REPO_SSH_KEY_PATH=$(realpath ./your_github_ssh_private_key) # If your GitHub ssh key isn't in "~.ssh/id_rsa"
56+
## Cleanup
4857

49-
cd app-tests
50-
./run.sh
58+
The script automatically cleans up all resources on exit. Manual cleanup:
59+
```bash
60+
docker compose -f docker-compose-app-tests.yml down -v
61+
rm -rf opal-tests-policy-repo temp-repo gitea-data git-repos .env
5162
```

‎app-tests/docker-compose-app-tests.yml

Lines changed: 45 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,46 @@
11
services:
2+
# Minimal Gitea for testing - no authentication, stateless
3+
gitea:
4+
image: gitea/gitea:latest-rootless
5+
container_name: gitea
6+
environment:
7+
- GITEA__server__DOMAIN=localhost
8+
- GITEA__server__HTTP_PORT=3000
9+
- GITEA__server__ROOT_URL=http://localhost:3000/
10+
- GITEA__server__OFFLINE_MODE=true
11+
# Security settings - disable all authentication
12+
- GITEA__security__INSTALL_LOCK=true
13+
- GITEA__security__SECRET_KEY=not-so-secret
14+
- GITEA__security__INTERNAL_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
15+
# Service settings - allow anonymous access
16+
- GITEA__service__DISABLE_REGISTRATION=true
17+
- GITEA__service__REQUIRE_SIGNIN_VIEW=false
18+
- GITEA__service__ENABLE_NOTIFY_MAIL=false
19+
- GITEA__service__ENABLE_CAPTCHA=false
20+
- GITEA__service__DEFAULT_ALLOW_CREATE_ORGANIZATION=true
21+
- GITEA__service__DEFAULT_ENABLE_DEPENDENCIES=false
22+
# Repository settings
23+
- GITEA__repository__ENABLE_PUSH_CREATE_USER=true
24+
- GITEA__repository__ENABLE_PUSH_CREATE_ORG=true
25+
- GITEA__repository__DEFAULT_BRANCH=main
26+
- GITEA__repository__DEFAULT_PRIVATE=false
27+
# Disable features we don't need
28+
- GITEA__webhook__ALLOWED_HOST_LIST=*
29+
- GITEA__migrations__ALLOWED_DOMAINS=*
30+
- GITEA__federation__ENABLED=false
31+
- GITEA__packages__ENABLED=false
32+
- GITEA__actions__ENABLED=false
33+
# Allow anonymous git operations
34+
- GITEA__repository__ENABLE_PUSH_CREATE_USER=true
35+
- GITEA__repository__ENABLE_PUSH_CREATE_ORG=true
36+
- GITEA__repository__FORCE_PRIVATE=false
37+
- GITEA__server__ENABLE_GZIP=false
38+
ports:
39+
- "3000:3000"
40+
- "2222:2222"
41+
networks:
42+
- default
43+
244
broadcast_channel:
345
image: postgres:alpine
446
environment:
@@ -15,9 +57,8 @@ services:
1557
environment:
1658
- OPAL_BROADCAST_URI=postgres://postgres:postgres@broadcast_channel:5432/postgres
1759
- UVICORN_NUM_WORKERS=4
18-
- OPAL_POLICY_REPO_URL=${OPAL_POLICY_REPO_URL:[email protected]:permitio/opal-tests-policy-repo.git}
19-
- OPAL_POLICY_REPO_MAIN_BRANCH=${POLICY_REPO_BRANCH}
20-
- OPAL_POLICY_REPO_SSH_KEY=${OPAL_POLICY_REPO_SSH_KEY}
60+
- OPAL_POLICY_REPO_URL=http://gitea:3000/gitea_admin/policy-repo.git
61+
- OPAL_POLICY_REPO_MAIN_BRANCH=${POLICY_REPO_BRANCH:-main}
2162
- OPAL_DATA_CONFIG_SOURCES={"config":{"entries":[{"url":"http://opal_server:7002/policy-data","config":{"headers":{"Authorization":"Bearer ${OPAL_CLIENT_TOKEN}"}},"topics":["policy_data"],"dst_path":"/static"}]}}
2263
- OPAL_LOG_FORMAT_INCLUDE_PID=true
2364
- OPAL_POLICY_REPO_WEBHOOK_SECRET=xxxxx
@@ -33,6 +74,7 @@ services:
3374
- "7002-7003:7002"
3475
depends_on:
3576
- broadcast_channel
77+
- gitea
3678

3779
opal_client:
3880
image: permitio/opal-client:${OPAL_IMAGE_TAG:-latest}

‎app-tests/opal-tests-policy-repo-main/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# opal-tests-policy-repo

‎app-tests/opal-tests-policy-repo-main/data.json

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
{
2+
"users": {
3+
"alice": {
4+
"roles": ["admin"],
5+
"location": {
6+
"country": "US",
7+
"ip": "8.8.8.8"
8+
}
9+
},
10+
"bob": {
11+
"roles": ["employee", "billing"],
12+
"location": {
13+
"country": "US",
14+
"ip": "8.8.8.8"
15+
}
16+
},
17+
"sunil": {
18+
"roles": ["guest"],
19+
"location": {
20+
"country": "US",
21+
"ip": "8.8.8.8"
22+
}
23+
},
24+
"eve": {
25+
"roles": ["customer"],
26+
"location": {
27+
"country": "US",
28+
"ip": "8.8.8.8"
29+
}
30+
}
31+
},
32+
"role_permissions": {
33+
"customer": [
34+
{
35+
"action": "read",
36+
"type": "dog"
37+
},
38+
{
39+
"action": "read",
40+
"type": "cat"
41+
},
42+
{
43+
"action": "adopt",
44+
"type": "dog"
45+
},
46+
{
47+
"action": "adopt",
48+
"type": "cat"
49+
}
50+
],
51+
"employee": [
52+
{
53+
"action": "read",
54+
"type": "dog"
55+
},
56+
{
57+
"action": "read",
58+
"type": "cat"
59+
},
60+
{
61+
"action": "update",
62+
"type": "dog"
63+
},
64+
{
65+
"action": "update",
66+
"type": "cat"
67+
}
68+
],
69+
"billing": [
70+
{
71+
"action": "read",
72+
"type": "finance"
73+
},
74+
{
75+
"action": "update",
76+
"type": "finance"
77+
}
78+
],
79+
"guest": [
80+
{
81+
"action": "read",
82+
"type": "cat"
83+
},
84+
{
85+
"action": "read",
86+
"type": "finance"
87+
}
88+
]
89+
}
90+
}

‎app-tests/opal-tests-policy-repo-main/policy.cedar

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
permit(
2+
principal in Role::"Editor",
3+
action in [
4+
Action::"document:read",
5+
Action::"document:write",
6+
Action::"document:delete"
7+
],
8+
resource in ResourceType::"document"
9+
);

‎app-tests/opal-tests-policy-repo-main/rbac.rego

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Role-based Access Control (RBAC)
2+
# --------------------------------
3+
#
4+
# This example defines an RBAC model for a Pet Store API. The Pet Store API allows
5+
# users to look at pets, adopt them, update their stats, and so on. The policy
6+
# controls which users can perform actions on which resources. The policy implements
7+
# a classic Role-based Access Control model where users are assigned to roles and
8+
# roles are granted the ability to perform some action(s) on some type of resource.
9+
#
10+
# This example shows how to:
11+
#
12+
# * Define an RBAC model in Rego that interprets role mappings represented in JSON.
13+
# * Iterate/search across JSON data structures (e.g., role mappings)
14+
#
15+
# For more information see:
16+
#
17+
# * Rego comparison to other systems: https://www.openpolicyagent.org/docs/latest/comparison-to-other-systems/
18+
# * Rego Iteration: https://www.openpolicyagent.org/docs/latest/#iteration
19+
20+
package app.rbac
21+
22+
# import data.utils
23+
24+
# By default, deny requests
25+
default allow = false
26+
27+
# Allow admins to do anything
28+
allow {
29+
user_is_admin
30+
}
31+
32+
# Allow bob to do anything
33+
#allow {
34+
# input.user == "bob"
35+
#}
36+
37+
# you can ignore this rule, it's simply here to create a dependency
38+
# to another rego policy file, so we can demonstate how to work with
39+
# an explicit manifest file (force order of policy loading).
40+
#allow {
41+
# input.matching_policy.grants
42+
# input.roles
43+
# utils.hasPermission(input.matching_policy.grants, input.roles)
44+
#}
45+
46+
# Allow the action if the user is granted permission to perform the action.
47+
allow {
48+
# Find permissions for the user.
49+
some permission
50+
user_is_granted[permission]
51+
52+
# Check if the permission permits the action.
53+
input.action == permission.action
54+
input.type == permission.type
55+
56+
# unless user location is outside US
57+
country := data.users[input.user].location.country
58+
country == "US"
59+
}
60+
61+
# user_is_admin is true if...
62+
user_is_admin {
63+
# for some `i`...
64+
some i
65+
66+
# "admin" is the `i`-th element in the user->role mappings for the identified user.
67+
data.users[input.user].roles[i] == "admin"
68+
}
69+
70+
# user_is_viewer is true if...
71+
user_is_viewer {
72+
# for some `i`...
73+
some i
74+
75+
# "viewer" is the `i`-th element in the user->role mappings for the identified user.
76+
data.users[input.user].roles[i] == "viewer"
77+
}
78+
79+
# user_is_guest is true if...
80+
user_is_guest {
81+
# for some `i`...
82+
some i
83+
84+
# "guest" is the `i`-th element in the user->role mappings for the identified user.
85+
data.users[input.user].roles[i] == "guest"
86+
}
87+
88+
89+
# user_is_granted is a set of permissions for the user identified in the request.
90+
# The `permission` will be contained if the set `user_is_granted` for every...
91+
user_is_granted[permission] {
92+
some i, j
93+
94+
# `role` assigned an element of the user_roles for this user...
95+
role := data.users[input.user].roles[i]
96+
97+
# `permission` assigned a single permission from the permissions list for 'role'...
98+
permission := data.role_permissions[role][j]
99+
}

‎app-tests/opal-tests-policy-repo-main/utils.rego

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
package utils
2+
hasPermission(grants, roles) {
3+
grants[_] == roles[_]
4+
}

0 commit comments

Comments
 (0)