-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add deployment test for Sentinel workflow
- Loading branch information
1 parent
4706c7c
commit 70781ad
Showing
3 changed files
with
144 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,144 @@ | ||
{ | ||
"cells": [ | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"# Sentinel Workflow \n", | ||
"\n", | ||
"The Sentinte harvesting workflow consists of two BPMN processes. The main process (Sentinel Registration Hourly) will be executed automatically be the Flowable engine every hour and searches for new data at CDSE. For each new scene discovered, the workflow executes another process (Sentinel Scene Ingestion) which performs the individual steps for harvesting and registering the data.\n", | ||
"\n", | ||
"This notebook demonstates the deployment of both processes at the Flowable instance in the EOEPCA Develop-Cluster and checks the next execution time of the main process." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"from requests import Session\n", | ||
"from requests.auth import HTTPBasicAuth\n", | ||
"import json\n", | ||
"\n", | ||
"flowable_base_url = \"https://registration-harvester-api.develop.eoepca.org/flowable-rest\"\n", | ||
"flowable_rest_user = \"eoepca\"\n", | ||
"flowable_rest_pw = \"eoepca\"\n", | ||
"bpmn_sentinel_registration_hourly = \"../workflows/sentinel-registration-hourly.bpmn\"\n", | ||
"bpmn_sentinel_scene_ingestion = \"../workflows/sentinel-scene-ingestion.bpmn\"\n", | ||
"\n", | ||
"flowable_session = Session()\n", | ||
"flowable_session.auth = HTTPBasicAuth(flowable_rest_user, flowable_rest_pw)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Deploy the BPMN processes" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"First, deploy the main process which searches for new Sentinel data at CDSE every hour." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"bpmn_file = {\"file\": open(bpmn_sentinel_registration_hourly, \"rb\")}\n", | ||
"response = flowable_session.post(f\"{flowable_base_url}/service/repository/deployments\", files=bpmn_file)\n", | ||
"print(json.dumps(response.json(), indent=4))" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"Then deploy the subprocess which implements the harvesting and registration of each scene discovered by the main process." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"bpmn_file = {\"file\": open(bpmn_sentinel_scene_ingestion, \"rb\")}\n", | ||
"response = flowable_session.post(f\"{flowable_base_url}/service/repository/deployments\", files=bpmn_file)\n", | ||
"print(json.dumps(response.json(), indent=4))" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"## Check processes and workflow execution timer" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"Both processes, named `Sentinel Registration Hourly` and `Sentinel Scene Ingestion` are now available at the Flowable instance." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"response = flowable_session.get(f\"{flowable_base_url}/service/repository/process-definitions\")\n", | ||
"processes = response.json()['data']\n", | ||
"for process in processes:\n", | ||
" print(f\"Process Definition: name='{process['name']}' id='{process['id']}'\")" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"The main process (Sentinel Registration Hourly) is supposed to start automatically each hour. A corresponding timer is started when the process is deployed. The datetime of the next workflow execution can be checked. The process id of the timer matches the id of the Sentinel Registration Hourly process." | ||
] | ||
}, | ||
{ | ||
"cell_type": "code", | ||
"execution_count": null, | ||
"metadata": {}, | ||
"outputs": [], | ||
"source": [ | ||
"response = flowable_session.get(f\"{flowable_base_url}/service/management/timer-jobs\")\n", | ||
"timer_jobs = response.json()['data']\n", | ||
"for job in timer_jobs:\n", | ||
" print(f\"Timer for process with id {job['processDefinitionId']} avaiable. Next execution at {job['dueDate']}.\")" | ||
] | ||
} | ||
], | ||
"metadata": { | ||
"kernelspec": { | ||
"display_name": ".venv", | ||
"language": "python", | ||
"name": "python3" | ||
}, | ||
"language_info": { | ||
"codemirror_mode": { | ||
"name": "ipython", | ||
"version": 3 | ||
}, | ||
"file_extension": ".py", | ||
"mimetype": "text/x-python", | ||
"name": "python", | ||
"nbconvert_exporter": "python", | ||
"pygments_lexer": "ipython3", | ||
"version": "3.12.7" | ||
} | ||
}, | ||
"nbformat": 4, | ||
"nbformat_minor": 2 | ||
} |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.