Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branch-3.0: [fix](local exchange) Use tokens to ensure that try_dequeue maintains strict order. #45741 #45844

Open
wants to merge 1 commit into
base: branch-3.0
Choose a base branch
from
Open

branch-3.0: [fix](local exchange) Use tokens to ensure that try_dequeue maintains strict order. #45741 #45844

wants to merge 1 commit into from
+112 −2

Conversation

github-actions[bot]
Copy link
Contributor

Cherry-picked from #45741

@Mryange
[fix](local exchange) Use tokens to ensure that try_dequeue maintains…
f6f160c 
… strict order. (#45741)

### What problem does this PR solve?

The previously used moodycamel::ConcurrentQueue does not guarantee that
the enqueue order matches the dequeue order,
even when there is only a single producer and a single consumer.  
Refer to this issue:
cameron314/concurrentqueue#316
We can use tokens to ensure the correct order.

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [x] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [x] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Dec 24, 2024
@hello-stephen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40843 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f6f160c21a2a7eb74504489b13fd3c2ea68a29e4, data reload: false

------ Round 1 ----------------------------------
q1	18517	7518	7350	7350
q2	2615	167	168	167
q3	11182	1140	1192	1140
q4	10367	726	695	695
q5	8043	2865	2808	2808
q6	233	145	161	145
q7	977	628	618	618
q8	9576	1966	1994	1966
q9	6917	6419	6389	6389
q10	7055	2324	2340	2324
q11	474	260	263	260
q12	404	213	213	213
q13	17785	2956	3020	2956
q14	245	222	206	206
q15	563	529	530	529
q16	669	612	618	612
q17	967	535	572	535
q18	7437	6721	6579	6579
q19	1378	1054	1051	1051
q20	479	202	220	202
q21	4196	3132	3293	3132
q22	1108	982	966	966
Total cold run time: 111187 ms
Total hot run time: 40843 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7212	7198	7209	7198
q2	326	229	233	229
q3	3186	2966	2854	2854
q4	2022	1833	1797	1797
q5	5671	5687	5622	5622
q6	220	142	152	142
q7	2250	1801	1793	1793
q8	3306	3477	3451	3451
q9	8846	8890	8866	8866
q10	3579	3592	3577	3577
q11	581	502	506	502
q12	822	576	567	567
q13	17170	3015	2984	2984
q14	295	258	268	258
q15	543	506	500	500
q16	712	676	652	652
q17	1814	1570	1553	1553
q18	7858	7382	7325	7325
q19	4568	1549	1604	1549
q20	2028	1790	1787	1787
q21	5247	5039	5071	5039
q22	1089	990	973	973
Total cold run time: 79345 ms
Total hot run time: 59218 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190291 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f6f160c21a2a7eb74504489b13fd3c2ea68a29e4, data reload: false

query1	998	363	358	358
query2	6508	2123	2078	2078
query3	6707	212	218	212
query4	34116	23610	23911	23610
query5	4354	440	435	435
query6	274	162	180	162
query7	4623	306	305	305
query8	276	218	220	218
query9	9588	2704	2683	2683
query10	485	261	251	251
query11	18433	15393	15601	15393
query12	154	101	99	99
query13	1662	428	402	402
query14	9637	6914	7401	6914
query15	246	172	173	172
query16	8038	499	478	478
query17	1619	579	554	554
query18	2022	314	314	314
query19	237	157	148	148
query20	112	109	109	109
query21	63	50	47	47
query22	4462	4051	4059	4051
query23	34225	33719	33454	33454
query24	12296	2865	2819	2819
query25	683	381	374	374
query26	1835	163	159	159
query27	2929	293	293	293
query28	8228	2464	2442	2442
query29	1009	435	407	407
query30	332	158	162	158
query31	1020	790	790	790
query32	89	57	56	56
query33	772	277	267	267
query34	1029	497	497	497
query35	859	712	747	712
query36	1115	930	933	930
query37	273	68	69	68
query38	3970	3786	3915	3786
query39	1474	1428	1435	1428
query40	229	78	79	78
query41	50	46	47	46
query42	111	93	97	93
query43	529	490	491	490
query44	1306	779	791	779
query45	180	164	166	164
query46	1156	706	700	700
query47	1915	1825	1810	1810
query48	458	362	368	362
query49	1290	376	373	373
query50	791	400	409	400
query51	7115	7134	7008	7008
query52	103	89	87	87
query53	254	185	184	184
query54	1218	434	432	432
query55	74	74	73	73
query56	256	239	233	233
query57	1207	1125	1104	1104
query58	231	197	202	197
query59	3174	2949	3003	2949
query60	295	256	246	246
query61	108	108	106	106
query62	919	733	721	721
query63	218	187	181	181
query64	5360	632	603	603
query65	3261	3215	3177	3177
query66	1430	315	331	315
query67	15616	15355	15379	15355
query68	3562	552	550	550
query69	406	275	273	273
query70	1147	1106	1127	1106
query71	323	253	249	249
query72	6354	4082	3890	3890
query73	762	348	357	348
query74	10126	9168	9091	9091
query75	3331	2647	2614	2614
query76	2589	972	997	972
query77	402	258	260	258
query78	10450	9819	9682	9682
query79	3437	588	580	580
query80	1865	413	405	405
query81	574	235	237	235
query82	907	113	111	111
query83	259	142	139	139
query84	296	79	76	76
query85	2283	305	290	290
query86	508	302	307	302
query87	4450	4408	4174	4174
query88	5077	2382	2370	2370
query89	426	305	301	301
query90	2069	183	185	183
query91	177	147	146	146
query92	67	48	50	48
query93	5025	540	525	525
query94	958	308	259	259
query95	363	248	250	248
query96	624	279	271	271
query97	3382	3114	3155	3114
query98	237	211	191	191
query99	1748	1391	1409	1391
Total cold run time: 309790 ms
Total hot run time: 190291 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@hello-stephen @doris-robot @dataroaring @Mryange