This repository has been archived by the owner on Dec 13, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #803 from bitbier/es-5-rest-module
Adds ES5 REST index DAO for conductor 1.12.x
- Loading branch information
Showing
17 changed files
with
2,126 additions
and
904 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,65 @@ | ||
| # ES5 Persistence | ||
|
|
||
| This module provides ES5 persistence when indexing workflows and tasks. | ||
|
|
||
| ## Usage | ||
|
|
||
| Set `workflow.elasticsearch.version=5` in Server module's configuration options. | ||
| This module uses the following configuration options: | ||
|
|
||
| * `workflow.elasticsearch.version` - This is the version of ES to use. | ||
| You must set this to `5` to use this module. | ||
| * `workflow.elasticsearch.transport` - This determines what type of transport to use. | ||
| Valid options are `tcp` or `rest`. | ||
| Default is `tcp`, if not specified. | ||
| * `workflow.elasticsearch.embedded` - This determines whether or not to run an embedded ES server when `db=memory`. | ||
| This option is ignored if `db` is set to some other value. | ||
| Default is `true`. | ||
| * `workflow.elasticsearch.url` - A comma separated list of schema/host/port of the ES nodes to communicate with. | ||
| Schema can be ignored when using `tcp` transport; otherwise, you must specify `http` or `https`. | ||
| If you are using `db=memory`, this will automatically use the correct host information to connect to it depending on transport. | ||
|
|
||
| * `workflow.elasticsearch.index.name` - The name of the workflow and task index. | ||
| Defaults to `conductor` | ||
| * `workflow.elasticsearch.tasklog.index.name` - The name of the task log index. | ||
| Defaults to `task_log` | ||
|
|
||
|
|
||
| ### REST Transport | ||
|
|
||
| If you are using AWS ElasticSearch, you should use the `rest` transport as that's the only version transport that they support. | ||
| However, this module currently only works with open IAM, VPC version of ElasticSearch. | ||
| Eventually, we should create ES modules that can be loaded in to support authentication and request signing, but this currently does not support that. | ||
|
|
||
| ### Example Configurations | ||
|
|
||
| **In-memory ES with TCP transport** | ||
|
|
||
| ``` | ||
| db=memory | ||
| workflow.elasticsearch.version=5 | ||
| ``` | ||
|
|
||
| **In-memory ES with REST transport** | ||
|
|
||
| ``` | ||
| db=memory | ||
| workflow.elasticsearch.version=5 | ||
| workflow.elasticsearch.transport=rest | ||
| ``` | ||
|
|
||
| **ES with TCP transport** | ||
|
|
||
| ``` | ||
| workflow.elasticsearch.version=5 | ||
| workflow.elasticsearch.url=127.0.0.1:9300 | ||
| ``` | ||
|
|
||
| **ES with REST transport** | ||
|
|
||
| ``` | ||
| workflow.elasticsearch.version=5 | ||
| workflow.elasticsearch.transport=rest | ||
| workflow.elasticsearch.url=http://127.0.0.1:9200 | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,32 @@ | ||
| dependencies { | ||
|
|
||
| compile project(':conductor-core') | ||
| compile (project(':conductor-core')) { | ||
| transitive = false | ||
| } | ||
| compile (project(':conductor-common')) { | ||
| transitive = false | ||
| } | ||
|
|
||
| // Specific 3rd parties we rely on | ||
| compile "com.google.inject:guice:${revGuice}" | ||
| compile "com.fasterxml.jackson.core:jackson-core:${revJacksonCore}" | ||
| compile "com.fasterxml.jackson.core:jackson-databind:${revJacksonDatabind}" | ||
| compile "com.google.inject:guice:${revGuice}" | ||
| compile "commons-io:commons-io:${revCommonsIo}" | ||
| compile "org.apache.commons:commons-lang3:${revCommonsLang3}" | ||
| compile "org.slf4j:slf4j-api:${revSlf4j}" | ||
| compile "joda-time:joda-time:${revJodaTime}" | ||
|
|
||
| //ES5 Dependency | ||
| compile "org.apache.logging.log4j:log4j-api:${revLog4jApi}" | ||
| compile "org.apache.logging.log4j:log4j-core:${revLog4jCore}" | ||
|
|
||
| compile "org.elasticsearch:elasticsearch:${revElasticSearch5}" | ||
| compile "org.elasticsearch.client:transport:${revElasticSearch5}" | ||
| compile "commons-io:commons-io:${revCommonsIo}" | ||
| compile "org.elasticsearch.client:elasticsearch-rest-client:${revElasticSearch5}" | ||
| compile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${revElasticSearch5}" | ||
|
|
||
| testRuntimeOnly "org.slf4j:slf4j-log4j12:${revSlf4jlog4j}" | ||
| testRuntimeOnly project(':conductor-core') | ||
| testRuntimeOnly project(':conductor-es5-persistence') | ||
| } |
Oops, something went wrong.