Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add integration test for Ceph RGW multisite sync policies feature #1193

Merged
merged 4 commits into from
May 31, 2024
Merged

Add integration test for Ceph RGW multisite sync policies feature #1193

merged 4 commits into from
May 31, 2024

Conversation

ionutbalutoiu
Copy link

Add Ceph RGW multisite sync policies integration test.

Add integration test for the Multisite sync policies feature proposed to the
upstream ceph-radosgw Juju charm:

@ionutbalutoiu ionutbalutoiu force-pushed the ceph-rgw-multisite-sync-policies branch from 5d96f9e to 0ab575e Compare March 19, 2024 14:12
@ionutbalutoiu
Copy link
Author

/cc @UtkarshBhatthere

@ionutbalutoiu ionutbalutoiu force-pushed the ceph-rgw-multisite-sync-policies branch 5 times, most recently from 8994bad to fd2fec1 Compare March 22, 2024 11:03
UtkarshBhatthere
UtkarshBhatthere suggested changes Apr 16, 2024
View reviewed changes
Copy link
Contributor

@UtkarshBhatthere UtkarshBhatthere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one comment.

zaza/openstack/charm_tests/ceph/tests.py Show resolved Hide resolved
@ionutbalutoiu ionutbalutoiu mentioned this pull request Apr 16, 2024
Open
UtkarshBhatthere
UtkarshBhatthere approved these changes Apr 23, 2024
View reviewed changes
Copy link
Contributor

@UtkarshBhatthere UtkarshBhatthere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

CephRGWTest: Reorder test cases
c0f0474 
* Rename `test_004_migration_and_multisite_failover` to `test_100_migration_and_multisite_failover`
    * This will allow us to insert more multi-site tests after `test_003`,
      before scale-down scenario is run in `test_100_migration_and_multisite_failover`.
* Rename `test_005_virtual_hosted_bucket` to `test_101_virtual_hosted_bucket`.
    * This was previously run after `test_004_migration_and_multisite_failover`.
      So, we rename the test case to `test_101` to keep the same order.

Signed-off-by: Ionut Balutoiu <[email protected]>
@ionutbalutoiu ionutbalutoiu force-pushed the ceph-rgw-multisite-sync-policies branch 2 times, most recently from 9d2bfc7 to c633240 Compare April 24, 2024 15:28
@ionutbalutoiu
Copy link
Author

LGTM

I rebased this PR against latest upstream main branch.

I noticed that a new test case (test_005_virtual_hosted_bucket) was added after test_004_migration_and_multisite_failover.

Besides renaming test_004_migration_and_multisite_failover to test_100_migration_and_multisite_failover as discussed previously (to have the multi-site scaledown scenario run last), I also renamed test_005_virtual_hosted_bucket to test_101_virtual_hosted_bucket to keep the same test order.

See commit CephRGWTest: Reorder test cases from this PR.

Besides the tests re-order, I also updated the new test case test_004_multisite_directional_sync_policy with my findings regarding the CI failures with focal-yoga-multisite (which installs Ceph Quincy release). More details in this comment from charm-ceph-radosgw patchset.

Ionut Balutoiu added 2 commits April 24, 2024 19:23
CephRGWTest: Wait for sites to be syncronised before recovery scenario
e68c902 
If `self.promote_rgw_to_primary(self.primary_rgw_app)` is executed
before sites are syncronised, the sites will not be syncronised after.

Signed-off-by: Ionut Balutoiu <[email protected]>
CephRGWTest: Add integration test for multisite sync policies feature
5054a79 
Signed-off-by: Ionut Balutoiu <[email protected]>
@ionutbalutoiu ionutbalutoiu force-pushed the ceph-rgw-multisite-sync-policies branch from c633240 to 5054a79 Compare April 24, 2024 16:24
@ionutbalutoiu
Copy link
Author

LGTM

I rebased this PR against latest upstream main branch.

I noticed that a new test case (test_005_virtual_hosted_bucket) was added after test_004_migration_and_multisite_failover.

Besides renaming test_004_migration_and_multisite_failover to test_100_migration_and_multisite_failover as discussed previously (to have the multi-site scaledown scenario run last), I also renamed test_005_virtual_hosted_bucket to test_101_virtual_hosted_bucket to keep the same test order.

See commit CephRGWTest: Reorder test cases from this PR.

Besides the tests re-order, I also updated the new test case test_004_multisite_directional_sync_policy with my findings regarding the CI failures with focal-yoga-multisite (which installs Ceph Quincy release). More details in this comment from charm-ceph-radosgw patchset.

Also, I included a small fix that might cause failures sometimes (it happened on my local Zaza env).

@ionutbalutoiu ionutbalutoiu force-pushed the ceph-rgw-multisite-sync-policies branch from a7cf06f to d2b38e6 Compare April 25, 2024 07:46
UtkarshBhatthere
UtkarshBhatthere approved these changes Apr 26, 2024
View reviewed changes
Copy link
Contributor

@UtkarshBhatthere UtkarshBhatthere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks for the tests @ionutbalutoiu

openstack-mirroring pushed a commit to openstack/openstack that referenced this pull request May 28, 2024
@openstack-gerrit
Update git submodules
78c0626 
* Update charm-ceph-radosgw from branch 'master'
  to 1f33f800fbfe58279e4e01aed344aa788a2a683b
  - Merge "Add group policy configuration"
  - Add group policy configuration
    
    Allow configuration of a zone group default sync policy. This is useful
    in scenarios where we want to have selective buckets sync. Valuable
    especially with the new `cloud-sync` relation.
    
    This is based on Ceph multisite sync policy:
    https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/
    
    Additionally, three more Juju actions are added to selectively enable,
    disable, or reset buckets sync:
    * `enable-buckets-sync`
    * `disable-buckets-sync`
    * `reset-buckets-sync`
    
    These new actions are meant to be used in conjunction with a default
    zone group sync policy that allows syncing, but it's disabled by default.
    
    Change-Id: I4a8076192269aaeaca50668ebcebc0a52c6d2c84
    func-test-pr: openstack-charmers/zaza-openstack-tests#1193
    Signed-off-by: Ionut Balutoiu <[email protected]>
openstack-mirroring pushed a commit to openstack/charm-ceph-radosgw that referenced this pull request May 28, 2024
Add group policy configuration
4e608c1 
Allow configuration of a zone group default sync policy. This is useful
in scenarios where we want to have selective buckets sync. Valuable
especially with the new `cloud-sync` relation.

This is based on Ceph multisite sync policy:
https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/

Additionally, three more Juju actions are added to selectively enable,
disable, or reset buckets sync:
* `enable-buckets-sync`
* `disable-buckets-sync`
* `reset-buckets-sync`

These new actions are meant to be used in conjunction with a default
zone group sync policy that allows syncing, but it's disabled by default.

Change-Id: I4a8076192269aaeaca50668ebcebc0a52c6d2c84
func-test-pr: openstack-charmers/zaza-openstack-tests#1193
Signed-off-by: Ionut Balutoiu <[email protected]>
@sabaini sabaini merged commit d4f7b08 into openstack-charmers:master May 31, 2024
3 checks passed
@sabaini
Copy link
Contributor

sabaini commented May 31, 2024

Thanks, lgtm! -- Merged

@ionutbalutoiu ionutbalutoiu deleted the ceph-rgw-multisite-sync-policies branch May 31, 2024 10:17
openstack-mirroring pushed a commit to openstack/charm-ceph-radosgw that referenced this pull request Sep 26, 2024
@UtkarshBhatthere
Add group policy configuration
12a2ebf 
Allow configuration of a zone group default sync policy. This is useful
in scenarios where we want to have selective buckets sync. Valuable
especially with the new `cloud-sync` relation.

This is based on Ceph multisite sync policy:
https://docs.ceph.com/en/latest/radosgw/multisite-sync-policy/

Additionally, three more Juju actions are added to selectively enable,
disable, or reset buckets sync:
* `enable-buckets-sync`
* `disable-buckets-sync`
* `reset-buckets-sync`

These new actions are meant to be used in conjunction with a default
zone group sync policy that allows syncing, but it's disabled by default.

Change-Id: I4a8076192269aaeaca50668ebcebc0a52c6d2c84
func-test-pr: openstack-charmers/zaza-openstack-tests#1193
Signed-off-by: Ionut Balutoiu <[email protected]>
(cherry picked from commit 4e608c1)
Signed-off-by: Utkarsh Bhatt <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@UtkarshBhatthere UtkarshBhatthere UtkarshBhatthere approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ionutbalutoiu @sabaini @UtkarshBhatthere