This repository has been archived by the owner on Dec 13, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Conductor UI Release - 3.3.0 (#2462)
* Squashed 3.0 UI release * Update top level README * Update docker README * Resolve typos and suggested changes in README & properties Co-authored-by: Peter Lau <[email protected]>
- Loading branch information
Showing
198 changed files
with
10,200 additions
and
23,343 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,8 @@ | ||
| server/build/libs | ||
| # General | ||
|
|
||
| # Backend | ||
| server/build/libs | ||
|
|
||
| # UI | ||
| **/node_modules | ||
| ui/build |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,27 +1,28 @@ | ||
| **/node_modules | ||
| **/npm-debug.log | ||
| ui/dist | ||
| ui/package-lock.json | ||
| # Java Build | ||
| .gradle | ||
| .project | ||
| .classpath | ||
| dump.rdb | ||
| out | ||
| bin | ||
| build | ||
| client/python/conductor.egg-info | ||
| target | ||
| buildscan.log | ||
| /docs/site | ||
|
|
||
| # Python | ||
| /polyglot-clients/python/conductor.egg-info | ||
| *.pyc | ||
| .classpath | ||
| docs/site | ||
| site | ||
| ui/.settings | ||
|
|
||
| # OS & IDE | ||
| .DS_Store | ||
| .settings | ||
| dump.rdb | ||
| .vscode | ||
| .idea | ||
| .project | ||
| *.iml | ||
| out/ | ||
| bin/ | ||
| target/ | ||
| .DS_Store | ||
| .vscode/ | ||
| buildscan.log | ||
|
|
||
| # JS & UI Related | ||
| node_modules | ||
| /ui/build | ||
|
|
||
| # publishing secrets | ||
| secrets/signing-key |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,76 +1,90 @@ | ||
| # Docker | ||
|
|
||
| This dockerfile runs using dynomite and elasticsearch. The conductor is split into the backend (server) and the frontend (ui). If an image with both of these items combined is desired, build the Dockerfile in the folder serverAndUI | ||
|
|
||
| ## Building the image | ||
| Dependency (build the jar files in ./server from project root) | ||
| - `gradlew build` | ||
|
|
||
| Building the images: | ||
| - `docker build -t conductor:server ./server` | ||
| - `docker build -t conductor:ui ./ui` | ||
|
|
||
| or using compose: | ||
| - `docker-compose build` | ||
|
|
||
| This builds the images: | ||
| - conductor:server - the conductor server and API. | ||
| - conductor:ui - the conductor frontend | ||
|
|
||
| ## Running conductor | ||
| Running the images: | ||
| - `docker run -p 8080:8080 -d -t conductor:server` | ||
| - `docker run -p 5000:5000 -d -t conductor:ui` (requires elasticsearch running locally) | ||
|
|
||
| Using compose (with Dynomite): | ||
| `docker-compose -f docker-compose.yaml -f docker-compose-dynomite.yaml up` | ||
|
|
||
| Using compose (with Postgres): | ||
| `docker-compose -f docker-compose.yaml -f docker-compose-postgres.yaml up` | ||
|
|
||
| ## Exiting Compose | ||
| `ctrl+c` will exit docker compose. | ||
|
|
||
| To ensure images are stopped do: | ||
| - `docker-compose down --remove-orphans` | ||
|
|
||
| ## Running in Interactive Mode | ||
| In interactive mode the default startup script for the container do not run | ||
| - `docker run -p 8080:8080 -t -i conductor:server -` | ||
| - `docker run -p 5000:5000 -t -i conductor:ui -` | ||
|
|
||
| ## Getting Started with Docker Compose | ||
| The easiest way to start experimenting with Conductor is via `docker-compose`. | ||
| ``` | ||
| cd docker | ||
| docker-compose build | ||
| docker-compose up | ||
| ``` | ||
|
|
||
| This default docker compose build establishes 3 services each running in its own container | ||
| * Elasticsearch | ||
| * Conductor Server | ||
| * Conductor UI | ||
|
|
||
| The UI can be accessed by pointing your browser to `http://localhost:5000/` | ||
| The Server API is accessible at `http://localhost:8080/` | ||
|
|
||
| ### Alternative Persistence Engines | ||
| By default `docker-compose.yaml` uses `config-local.properties`. This configures the `memory` database, where data is lost when the server terminates. This configuration is useful for testing or demo only. | ||
|
|
||
| A selection of `docker-compose-*.yaml` and `config-*.properties` files are provided demonstrating the use of alternative persistence engines. | ||
|
|
||
| For example this will start the server instance backed by a PostgreSQL DB. | ||
| ``` | ||
| docker-compose -f docker-compose.yaml -f docker-compose-postgres.yaml up | ||
| ``` | ||
|
|
||
| ### Exiting Compose | ||
| `Ctrl+c` will exit docker compose. | ||
|
|
||
| To ensure images are stopped execute: `docker-compose down`. | ||
|
|
||
| ## Standalone Server Image | ||
| To build and run the server image, without using `docker-compose`, from the `docker` directory execute: | ||
| ``` | ||
| docker build -t conductor:server -f server/Dockerfile ../ | ||
| docker run -p 8080:8080 -d --name conductor_server conductor:server | ||
| ``` | ||
| This builds the image `conductor:server` and runs it in a container named `conductor_server`. The API should now be accessible at `localhost:8080`. | ||
|
|
||
| To 'login' to the running container, use the command: | ||
| ``` | ||
| docker exec -it conductor_server /bin/sh | ||
| ``` | ||
|
|
||
| ## Standalone UI Image | ||
| From the `docker` directory, | ||
| ``` | ||
| docker build -t conductor:ui -f ui/Dockerfile ../ | ||
| docker run -p 5000:5000 -d --name conductor_ui conductor:ui | ||
| ``` | ||
| This builds the image `conductor:ui` and runs it in a container named `conductor_ui`. The UI should now be accessible at `localhost:5000`. | ||
|
|
||
| ### Note | ||
| * In order for the UI to do anything useful the Conductor Server must already be running on port 8080, either in a Docker container (see above), or running directly in the local JRE. | ||
| * Additionally, significant parts of the UI will not be functional without Elastisearch being available. Using the `docker-compose` approach alleviates these considerations. | ||
|
|
||
| ## Monitoring with Prometheus | ||
|
|
||
| Start Prometheus with: | ||
|
|
||
| `docker-compose -f docker-compose-prometheus.yaml up -d` | ||
|
|
||
| Go to [http://127.0.0.1:9090](http://127.0.0.1:9090). | ||
|
|
||
|
|
||
| ## Potential problem when using docker compose | ||
| ## Potential problem when using Docker Images | ||
|
|
||
| #### Elasticsearch timeout | ||
| Standalone(single node) elasticsearch has a yellow status which will cause timeout for conductor server (Required: Green). | ||
| Spin up a cluster (more than one) to prevent timeout or use config option `conductor.elasticsearch.clusteHealthColor=yellow`. | ||
|
|
||
| Elasticsearch timeout | ||
| Standalone(single node) elasticsearch has a yellow status which will cause timeout for conductor server(Required: Green). | ||
| Spin up a cluster(More than one) to prevent timeout or edit the local code(check the issue tagged for more) | ||
| Check issue: https://github.com/Netflix/conductor/issues/2262 | ||
| See issue: https://github.com/Netflix/conductor/issues/2262 | ||
|
|
||
| Changes does not reflect after changes in config.properties | ||
| #### Changes in config-*.properties do not take effect | ||
| Config is copy into image during docker build. You have to rebuild the image or better, link a volume to it to reflect new changes. | ||
|
|
||
| To troubleshoot a failed startup | ||
| Check the log of the server, which is located at app/logs (default directory in dockerfile) | ||
| #### To troubleshoot a failed startup | ||
| Check the log of the server, which is located at `/app/logs` (default directory in dockerfile) | ||
|
|
||
| Unable to access to conductor:server with rest | ||
| #### Unable to access to conductor:server API on port 8080 | ||
| It may takes some time for conductor server to start. Please check server log for potential error. | ||
| issue: https://github.com/Netflix/conductor/issues/1725#issuecomment-651806800 | ||
|
|
||
| How to disable elasticsearch | ||
| #### How to disable elasticsearch | ||
| Elasticsearch is optional, please be aware that disable it will make most of the conductor UI not functional. | ||
| Set `workflow.indexing.enabled=false` in your_config.properties | ||
| Comment out all the config related to elasticsearch | ||
| eg: `conductor.elasticsearch.url=http://es:9200` | ||
| Pull request: https://github.com/Netflix/conductor/pull/1555#issue-382145486 | ||
|
|
||
| * Set `workflow.indexing.enabled=false` in your_config.properties | ||
| * Comment out all the config related to elasticsearch | ||
| E.g.: `conductor.elasticsearch.url=http://es:9200` | ||
|
|
||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,10 +2,12 @@ | |
| # conductor:server - Netflix conductor server | ||
| # | ||
|
|
||
| # =========================================================================================================== | ||
| # 0. Builder stage | ||
| # =========================================================================================================== | ||
| FROM openjdk:11-jdk AS builder | ||
|
|
||
| MAINTAINER Netflix OSS <[email protected]> | ||
| LABEL maintainer="Netflix OSS <[email protected]>" | ||
|
|
||
| # Copy the project directly onto the image | ||
| COPY . /conductor | ||
|
|
@@ -14,25 +16,24 @@ WORKDIR /conductor | |
| # Build the server on run | ||
| RUN ./gradlew build -x test | ||
|
|
||
| # =========================================================================================================== | ||
| # 1. Bin stage | ||
| # =========================================================================================================== | ||
| FROM openjdk:11-jre | ||
|
|
||
| MAINTAINER Netflix OSS <[email protected]> | ||
| LABEL maintainer="Netflix OSS <[email protected]>" | ||
|
|
||
| # Make app folders | ||
| RUN mkdir -p /app/config /app/logs /app/libs | ||
|
|
||
| # Copy the project directly onto the image | ||
| # Copy the compiled output to new image | ||
| COPY --from=builder /conductor/docker/server/bin /app | ||
| COPY --from=builder /conductor/docker/server/config /app/config | ||
| COPY --from=builder /conductor/server/build/libs/conductor-server-*-boot.jar /app/libs | ||
|
|
||
| # Copy the files for the server into the app folders | ||
| RUN chmod +x /app/startup.sh | ||
|
|
||
| EXPOSE 8080 | ||
| EXPOSE 8090 | ||
|
|
||
| HEALTHCHECK --interval=60s --timeout=30s --retries=10 CMD curl -I -XGET http://localhost:8080/health || exit 1 | ||
|
|
||
| CMD [ "/app/startup.sh" ] | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.