Skip to content

[fix](local shuffle) fix bucket shuffle + set operation + local shuffle compute wrong result#60823

Merged
924060929 merged 1 commit intoapache:masterfrom
924060929:master-fix-bucket-set-operation
Feb 26, 2026
Merged

[fix](local shuffle) fix bucket shuffle + set operation + local shuffle compute wrong result#60823
924060929 merged 1 commit intoapache:masterfrom
924060929:master-fix-bucket-set-operation

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Feb 25, 2026

What problem does this PR solve?

fix bucket shuffle + set operation + local shuffle compute wrong result, because backend can not plan the correct local shuffle type, introduced by #59006.
so we should disable bucket shuffle for set operation now, after support plan local shuffle in frontend, we can support this feature.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 28953 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aac283d0d12609f2837a99ae96ed0659bc583bcb, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17638	4459	4328	4328
q2	q3	10655	778	531	531
q4	4675	355	255	255
q5	7558	1200	1010	1010
q6	171	175	148	148
q7	790	850	677	677
q8	9307	1469	1342	1342
q9	4799	4706	4616	4616
q10	6772	1869	1642	1642
q11	456	255	254	254
q12	718	569	470	470
q13	17768	4195	3436	3436
q14	234	235	209	209
q15	925	787	788	787
q16	751	714	687	687
q17	733	854	409	409
q18	6087	5344	5300	5300
q19	1244	987	643	643
q20	510	484	390	390
q21	4898	2051	1531	1531
q22	353	304	288	288
Total cold run time: 97042 ms
Total hot run time: 28953 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4622	4516	4547	4516
q2	q3	1801	2200	1772	1772
q4	860	1193	784	784
q5	4049	4376	4331	4331
q6	196	182	145	145
q7	1779	1728	1526	1526
q8	2665	2702	2597	2597
q9	7450	7329	7292	7292
q10	2616	2833	2391	2391
q11	523	442	413	413
q12	491	598	447	447
q13	4193	4453	3628	3628
q14	287	298	282	282
q15	892	796	834	796
q16	728	783	710	710
q17	1171	1507	1336	1336
q18	7087	6757	6583	6583
q19	1056	881	868	868
q20	2062	2144	2005	2005
q21	3969	3433	3350	3350
q22	533	462	411	411
Total cold run time: 49030 ms
Total hot run time: 46183 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183904 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aac283d0d12609f2837a99ae96ed0659bc583bcb, data reload: false

query5	5202	641	518	518
query6	318	223	206	206
query7	4220	458	266	266
query8	329	232	228	228
query9	8740	2697	2693	2693
query10	539	367	347	347
query11	17115	16856	16624	16624
query12	190	130	124	124
query13	1270	437	364	364
query14	6705	3169	2916	2916
query14_1	2806	2857	2780	2780
query15	197	188	177	177
query16	979	461	461	461
query17	1061	692	577	577
query18	2701	443	351	351
query19	227	198	175	175
query20	141	126	126	126
query21	226	146	121	121
query22	5355	5574	5545	5545
query23	17439	17139	16841	16841
query23_1	17354	16745	16660	16660
query24	7152	1618	1214	1214
query24_1	1231	1224	1237	1224
query25	568	495	421	421
query26	1240	264	150	150
query27	2766	466	290	290
query28	4539	1828	1845	1828
query29	814	577	496	496
query30	312	251	213	213
query31	868	729	641	641
query32	84	73	70	70
query33	526	354	299	299
query34	925	920	572	572
query35	641	707	595	595
query36	1085	1140	1037	1037
query37	136	94	87	87
query38	2930	2894	2864	2864
query39	906	880	847	847
query39_1	830	830	821	821
query40	230	155	139	139
query41	67	65	62	62
query42	107	104	104	104
query43	368	379	363	363
query44	
query45	202	197	186	186
query46	880	995	614	614
query47	2098	2141	2057	2057
query48	317	318	230	230
query49	637	488	431	431
query50	677	279	213	213
query51	4104	4105	4064	4064
query52	105	109	95	95
query53	289	347	294	294
query54	303	261	255	255
query55	89	89	78	78
query56	298	303	295	295
query57	1397	1322	1288	1288
query58	285	286	271	271
query59	2613	2813	2616	2616
query60	328	330	306	306
query61	153	150	143	143
query62	637	581	544	544
query63	310	273	275	273
query64	4874	1274	1002	1002
query65	
query66	1422	450	351	351
query67	16385	16377	16271	16271
query68	
query69	383	313	282	282
query70	976	993	941	941
query71	337	306	294	294
query72	2831	2653	2504	2504
query73	531	544	316	316
query74	10026	9941	9789	9789
query75	2853	2772	2436	2436
query76	2283	1069	667	667
query77	357	376	312	312
query78	11220	11566	10787	10787
query79	2721	785	605	605
query80	1874	617	553	553
query81	564	274	244	244
query82	999	151	114	114
query83	349	272	242	242
query84	254	123	102	102
query85	973	478	445	445
query86	454	314	326	314
query87	3094	3082	3005	3005
query88	3525	2657	2651	2651
query89	434	368	343	343
query90	1942	171	168	168
query91	169	157	136	136
query92	78	84	68	68
query93	1634	836	493	493
query94	656	347	288	288
query95	575	337	376	337
query96	652	521	234	234
query97	2472	2498	2399	2399
query98	236	215	222	215
query99	963	979	890	890
Total cold run time: 257813 ms
Total hot run time: 183904 ms

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (11/11) 🎉
Increment coverage report
Complete coverage report

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 26, 2026
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit 5310494 into apache:master Feb 26, 2026
32 of 34 checks passed
@924060929 924060929 deleted the master-fix-bucket-set-operation branch February 26, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants