Skip to content

Commit

Permalink
Add deployment test for Sentinel workflow
Browse files Browse the repository at this point in the history
  • Loading branch information
mario-winkler committed Oct 28, 2024
1 parent 4706c7c commit 70781ad
Show file tree
Hide file tree
Showing 3 changed files with 144 additions and 0 deletions.
144 changes: 144 additions & 0 deletions tests/test-sentinel-workflow.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Sentinel Workflow \n",
"\n",
"The Sentinte harvesting workflow consists of two BPMN processes. The main process (Sentinel Registration Hourly) will be executed automatically be the Flowable engine every hour and searches for new data at CDSE. For each new scene discovered, the workflow executes another process (Sentinel Scene Ingestion) which performs the individual steps for harvesting and registering the data.\n",
"\n",
"This notebook demonstates the deployment of both processes at the Flowable instance in the EOEPCA Develop-Cluster and checks the next execution time of the main process."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from requests import Session\n",
"from requests.auth import HTTPBasicAuth\n",
"import json\n",
"\n",
"flowable_base_url = \"https://registration-harvester-api.develop.eoepca.org/flowable-rest\"\n",
"flowable_rest_user = \"eoepca\"\n",
"flowable_rest_pw = \"eoepca\"\n",
"bpmn_sentinel_registration_hourly = \"../workflows/sentinel-registration-hourly.bpmn\"\n",
"bpmn_sentinel_scene_ingestion = \"../workflows/sentinel-scene-ingestion.bpmn\"\n",
"\n",
"flowable_session = Session()\n",
"flowable_session.auth = HTTPBasicAuth(flowable_rest_user, flowable_rest_pw)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Deploy the BPMN processes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, deploy the main process which searches for new Sentinel data at CDSE every hour."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"bpmn_file = {\"file\": open(bpmn_sentinel_registration_hourly, \"rb\")}\n",
"response = flowable_session.post(f\"{flowable_base_url}/service/repository/deployments\", files=bpmn_file)\n",
"print(json.dumps(response.json(), indent=4))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then deploy the subprocess which implements the harvesting and registration of each scene discovered by the main process."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"bpmn_file = {\"file\": open(bpmn_sentinel_scene_ingestion, \"rb\")}\n",
"response = flowable_session.post(f\"{flowable_base_url}/service/repository/deployments\", files=bpmn_file)\n",
"print(json.dumps(response.json(), indent=4))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Check processes and workflow execution timer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Both processes, named `Sentinel Registration Hourly` and `Sentinel Scene Ingestion` are now available at the Flowable instance."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = flowable_session.get(f\"{flowable_base_url}/service/repository/process-definitions\")\n",
"processes = response.json()['data']\n",
"for process in processes:\n",
" print(f\"Process Definition: name='{process['name']}' id='{process['id']}'\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The main process (Sentinel Registration Hourly) is supposed to start automatically each hour. A corresponding timer is started when the process is deployed. The datetime of the next workflow execution can be checked. The process id of the timer matches the id of the Sentinel Registration Hourly process."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"response = flowable_session.get(f\"{flowable_base_url}/service/management/timer-jobs\")\n",
"timer_jobs = response.json()['data']\n",
"for job in timer_jobs:\n",
" print(f\"Timer for process with id {job['processDefinitionId']} avaiable. Next execution at {job['dueDate']}.\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Binary file added workflows/sentinel-registration-hourly.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added workflows/sentinel-scene-ingestion.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 70781ad

Please sign in to comment.