generated from canonical/template-operator
-
Notifications
You must be signed in to change notification settings - Fork 8
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[DPE-3661] Add support for large deployments backup (#248)
Implements [DPE-3661](https://warthogs.atlassian.net/browse/DPE-3661): extend backup feature to support large deployment scenarios. Currently, there are 4x type of scenarios: 1) Small deployments: the cluster performs all the different types of node roles 2) Large deployments - orchestrator: the app is in charge of not only its own application units but also to coordinate the across the different clusters 3) Large deployments - failover orchestrator: very similar to (2), this app must also publish its information in the peer relation, although all the clusters will only listen to the active manager 4) Large deployments - data only: do not perform any management tasks and should receive any relevant information via peer relation For backups, clusters of type (3) and (4) have a special behavior: they will receive the backup data via peer-cluster relation and should refuse: i. to execute backup-related actions; and ii. to execute the s3-relation events themselves. The latter avoids confusions, e.g. an user inadvertently relates the cluster to different s3-integrators. The implementation of (1) and (2) are very similar. It contains the same fix as: #253 Adds following fixes related to testing in general: 1. `ContinuousWrites` is updated to hold the right count of documents in `writes_value` internally 2. Adds an `is_burst` option to `ContinuousWrites`: a test may choose to send 100-burst docs vs. doc-by-doc - `is_burst` defaults to `True` 3. The `ContinuousWrites` terminates its process as part of `stop`, avoiding stranded process generating docs to `ContinuousWrites.INDEX_NAME` post a given test 4. The `start_and_check_continuous_writes` updated to `assert_start_and_check_continuous_writes` # Implementation Details For developers, there is no meaningful difference between small and large deployments. They both use the same backup_factory() to return the correct object for their case. The large deployments expands the original concept of OpenSearchBackup to include other juju applications that are not cluster_manager. This means a cluster may be a data-only or even a failover cluster-manager and still interacts with s3-integrator at a certain level. The baseline is that every unit in the cluster must import the S3 credentials. The main orchestrator will share these credentials via the peer-cluster relation. Failover and data clusters will import that information from the peer-cluster relation. To implement the points above without causing too much disruption to the existing code, a factory pattern has been adopted, where the main charm receives a OpenSearchBackupBase object that corresponds to its own case (cluster-manager, failover, data, etc). """ [DPE-3661]: https://warthogs.atlassian.net/browse/DPE-3661?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
- Loading branch information
1 parent
c2c19f2
commit 3a85973
Showing
14 changed files
with
616 additions
and
100 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.