[fix](cloud-mow) MS should create new Transaction to continue geting delete bitmap when encounter TXN_TOO_OLD #43509
Conversation
…delete bitmap when encounter TXN_TOO_OLD
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
| do { | ||
| err = txn->get(start_key, end_key, &it); | ||
| #ifdef UNIT_TEST |
There was a problem hiding this comment.
use syncpoint instead of MACRO, or change config::max_get_delete_bitmap_retry_times before testing
int n = 10000;
syncpoint(&n);
err = txn->get(start_key, end_key, &it, true, n);
| @@ -211,6 +211,9 @@ CONF_Validator(s3_client_http_scheme, [](const std::string& config) -> bool { | |||
| // Max retry times for object storage request | |||
| CONF_mInt64(max_s3_client_retry, "10"); | |||
|
|
|||
| // Max retry times for get delete bitmap | |||
| CONF_mInt64(max_get_delete_bitmap_retry_times, "120"); | |||
There was a problem hiding this comment.
suggest naming max_get_delete_bitmap_iterations
| @@ -211,6 +211,9 @@ CONF_Validator(s3_client_http_scheme, [](const std::string& config) -> bool { | |||
| // Max retry times for object storage request | |||
| CONF_mInt64(max_s3_client_retry, "10"); | |||
|
|
|||
| // Max retry times for get delete bitmap | |||
| CONF_mInt64(max_get_delete_bitmap_retry_times, "120"); | |||
There was a problem hiding this comment.
Do we really need to retry so many times?
| return; | ||
| } | ||
| #ifdef UNIT_TEST | ||
| err = txn->get(start_key, end_key, &it, true, 2); |
There was a problem hiding this comment.
make sure all delete bitmap keys are committed before rowset keys, so that the snapshot read won't miss data.
| #else | ||
| err = txn->get(start_key, end_key, &it, true); | ||
| #endif | ||
| retry++; |
There was a problem hiding this comment.
print the delete bitmap count and size once you have retry, which means there are too many delete bitmaps ,we may need log to debug.
When delete bitmap count is big, geting delete bitmap may encounter TXN_TOO_OLD, ms should create a new transaction to reading the remaining data instead of returning TXN_TOO_OLD code.
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)