Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test] (BE-UT)Add distinct agg hashmap be ut #47291

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Mryange
Copy link
Contributor

@Mryange Mryange commented Jan 21, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange
Copy link
Contributor Author

Mryange commented Jan 21, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32440 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 67ab981d1e4feb523b409895b07cca19986f35a5, data reload: false

------ Round 1 ----------------------------------
q1	17633	5585	5398	5398
q2	2053	306	179	179
q3	10397	1273	741	741
q4	10239	968	518	518
q5	7534	2423	2189	2189
q6	196	172	138	138
q7	920	781	592	592
q8	9243	1384	1172	1172
q9	5290	4940	4901	4901
q10	6841	2360	1908	1908
q11	462	277	265	265
q12	352	361	220	220
q13	17800	3724	3138	3138
q14	231	245	224	224
q15	521	496	466	466
q16	634	612	577	577
q17	558	867	329	329
q18	7047	6494	6410	6410
q19	1203	957	537	537
q20	315	328	195	195
q21	3060	2282	2030	2030
q22	362	342	313	313
Total cold run time: 102891 ms
Total hot run time: 32440 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5555	5496	5559	5496
q2	244	331	242	242
q3	2310	2655	2315	2315
q4	1445	1944	1409	1409
q5	4357	4715	4721	4715
q6	164	162	127	127
q7	2115	1994	1880	1880
q8	2658	2869	2696	2696
q9	7309	7300	7373	7300
q10	2916	3271	2823	2823
q11	582	518	507	507
q12	687	770	606	606
q13	3503	4008	3309	3309
q14	293	311	267	267
q15	520	473	467	467
q16	664	711	632	632
q17	1261	1767	1248	1248
q18	7662	7562	7357	7357
q19	776	1045	1161	1045
q20	2017	2123	1815	1815
q21	5924	5403	5113	5113
q22	633	630	596	596
Total cold run time: 53595 ms
Total hot run time: 51965 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194888 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 67ab981d1e4feb523b409895b07cca19986f35a5, data reload: false

query1	1329	967	956	956
query2	6259	2043	2061	2043
query3	11164	4797	4705	4705
query4	60588	34186	23646	23646
query5	5600	618	488	488
query6	344	199	174	174
query7	5063	498	292	292
query8	305	232	233	232
query9	6499	2580	2600	2580
query10	432	320	258	258
query11	15528	15099	15649	15099
query12	161	112	110	110
query13	1068	572	439	439
query14	11135	7216	7876	7216
query15	222	212	190	190
query16	6647	642	489	489
query17	1130	758	596	596
query18	1792	421	351	351
query19	187	186	161	161
query20	115	115	107	107
query21	208	120	104	104
query22	4470	4543	4505	4505
query23	34113	33438	33546	33438
query24	5676	2271	2329	2271
query25	520	462	397	397
query26	750	280	157	157
query27	2406	483	332	332
query28	4145	2488	2420	2420
query29	544	557	434	434
query30	210	192	158	158
query31	943	929	818	818
query32	67	60	55	55
query33	427	371	304	304
query34	746	902	524	524
query35	832	852	760	760
query36	1011	1057	955	955
query37	124	130	79	79
query38	4391	4262	4256	4256
query39	1516	1471	1450	1450
query40	209	113	101	101
query41	49	53	49	49
query42	121	105	110	105
query43	517	525	494	494
query44	1359	836	862	836
query45	210	176	180	176
query46	876	1071	673	673
query47	1930	1925	1898	1898
query48	382	418	331	331
query49	713	484	411	411
query50	660	695	413	413
query51	7103	7070	6838	6838
query52	104	95	92	92
query53	229	251	189	189
query54	473	486	416	416
query55	80	86	77	77
query56	258	262	253	253
query57	1185	1140	1058	1058
query58	235	232	225	225
query59	2963	3079	2952	2952
query60	283	263	255	255
query61	141	112	113	112
query62	708	703	615	615
query63	223	191	188	188
query64	1746	984	648	648
query65	3263	3193	3164	3164
query66	732	416	304	304
query67	15901	15618	15361	15361
query68	4761	821	528	528
query69	500	295	268	268
query70	1190	1158	1079	1079
query71	396	303	268	268
query72	5991	3804	3813	3804
query73	785	754	351	351
query74	10052	9122	8675	8675
query75	3222	3160	2669	2669
query76	3641	1197	777	777
query77	503	363	285	285
query78	10084	9939	9356	9356
query79	3616	826	597	597
query80	1705	534	440	440
query81	560	285	238	238
query82	356	163	128	128
query83	271	172	152	152
query84	283	91	86	86
query85	768	350	303	303
query86	469	322	302	302
query87	4493	4516	4413	4413
query88	4216	2182	2160	2160
query89	406	322	288	288
query90	1556	191	197	191
query91	136	196	105	105
query92	67	56	54	54
query93	2752	854	538	538
query94	765	415	285	285
query95	336	270	253	253
query96	488	621	286	286
query97	2762	2881	2738	2738
query98	220	204	194	194
query99	1287	1379	1294	1294
Total cold run time: 309957 ms
Total hot run time: 194888 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.73% (10876/26062)
Line Coverage: 32.09% (91946/286499)
Region Coverage: 31.21% (47070/150830)
Branch Coverage: 27.31% (23868/87390)
Coverage Report: http://coverage.selectdb-in.cc/coverage/67ab981d1e4feb523b409895b07cca19986f35a5_67ab981d1e4feb523b409895b07cca19986f35a5/report/index.html

@doris-robot
Copy link

ClickBench: Total hot run time: 31.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 67ab981d1e4feb523b409895b07cca19986f35a5, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.61	0.10	0.11
query5	0.41	0.43	0.43
query6	1.14	0.67	0.68
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.58	0.52	0.50
query10	0.56	0.57	0.55
query11	0.15	0.10	0.11
query12	0.14	0.11	0.11
query13	0.64	0.61	0.62
query14	2.70	2.76	2.73
query15	0.94	0.86	0.86
query16	0.40	0.38	0.40
query17	1.03	1.04	1.06
query18	0.23	0.22	0.21
query19	2.01	1.89	2.06
query20	0.01	0.02	0.02
query21	15.36	0.94	0.57
query22	0.76	0.91	0.66
query23	15.23	1.52	0.63
query24	2.95	1.30	0.70
query25	0.26	0.17	0.15
query26	0.28	0.14	0.14
query27	0.05	0.05	0.05
query28	13.39	1.08	0.45
query29	12.58	4.01	3.24
query30	0.25	0.10	0.07
query31	2.81	0.61	0.42
query32	3.23	0.58	0.48
query33	3.11	3.06	3.06
query34	16.33	5.37	4.78
query35	4.74	4.66	4.66
query36	0.66	0.50	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.02
query40	0.18	0.13	0.12
query41	0.07	0.03	0.03
query42	0.04	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.44 s
Total hot run time: 31.24 s

@Mryange
Copy link
Contributor Author

Mryange commented Jan 22, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32377 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6700cf92defd8af885db6045894a78250db1cd8c, data reload: false

------ Round 1 ----------------------------------
q1	17569	5462	5399	5399
q2	2047	324	178	178
q3	10390	1213	734	734
q4	10213	961	532	532
q5	7558	2369	2148	2148
q6	191	165	132	132
q7	905	763	595	595
q8	9255	1344	1172	1172
q9	5286	4983	4886	4886
q10	6829	2337	1885	1885
q11	473	290	261	261
q12	340	353	215	215
q13	17770	3658	3174	3174
q14	229	221	209	209
q15	510	463	468	463
q16	621	611	590	590
q17	553	847	326	326
q18	6971	6364	6383	6364
q19	1323	942	561	561
q20	310	336	189	189
q21	2962	2251	2055	2055
q22	370	349	309	309
Total cold run time: 102675 ms
Total hot run time: 32377 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5517	5531	5558	5531
q2	248	328	245	245
q3	2269	2660	2310	2310
q4	1421	1881	1419	1419
q5	4317	4829	4623	4623
q6	171	159	126	126
q7	2068	1999	1853	1853
q8	2661	2845	2735	2735
q9	7637	7403	7447	7403
q10	3123	3406	2877	2877
q11	587	513	509	509
q12	693	750	622	622
q13	3619	4082	3344	3344
q14	280	303	278	278
q15	525	495	479	479
q16	655	697	675	675
q17	1250	1779	1282	1282
q18	7726	7647	7378	7378
q19	852	1165	1121	1121
q20	2034	2020	1900	1900
q21	5931	5189	5462	5189
q22	674	664	598	598
Total cold run time: 54258 ms
Total hot run time: 52497 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.71% (10873/26070)
Line Coverage: 32.08% (91924/286526)
Region Coverage: 31.20% (47065/150843)
Branch Coverage: 27.31% (23862/87390)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6700cf92defd8af885db6045894a78250db1cd8c_6700cf92defd8af885db6045894a78250db1cd8c/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 193640 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6700cf92defd8af885db6045894a78250db1cd8c, data reload: false

query1	2465	1048	947	947
query2	7553	1982	1936	1936
query3	13747	4485	4670	4485
query4	32589	23503	22975	22975
query5	4530	588	447	447
query6	283	204	198	198
query7	3996	508	306	306
query8	301	249	238	238
query9	9485	2600	2553	2553
query10	476	308	251	251
query11	17898	15135	14967	14967
query12	161	109	106	106
query13	1638	532	387	387
query14	10965	6898	7046	6898
query15	254	210	177	177
query16	7910	636	515	515
query17	1520	753	561	561
query18	2081	421	290	290
query19	198	191	159	159
query20	117	112	115	112
query21	211	135	101	101
query22	4487	4721	4341	4341
query23	34175	33621	33113	33113
query24	6429	2277	2361	2277
query25	482	448	391	391
query26	783	289	152	152
query27	2051	476	331	331
query28	5163	2461	2452	2452
query29	647	563	418	418
query30	218	190	160	160
query31	969	935	831	831
query32	84	61	53	53
query33	469	370	311	311
query34	787	898	504	504
query35	849	833	757	757
query36	1025	1043	965	965
query37	122	104	81	81
query38	4357	4331	4276	4276
query39	1543	1484	1423	1423
query40	200	115	106	106
query41	51	51	50	50
query42	121	101	97	97
query43	518	508	495	495
query44	1389	828	832	828
query45	186	172	171	171
query46	883	1066	642	642
query47	1893	1898	1867	1867
query48	385	397	343	343
query49	730	506	401	401
query50	707	665	404	404
query51	7053	7116	6939	6939
query52	110	106	92	92
query53	227	261	186	186
query54	500	494	415	415
query55	87	84	81	81
query56	261	252	232	232
query57	1239	1231	1110	1110
query58	255	235	256	235
query59	3143	3134	3090	3090
query60	284	283	266	266
query61	143	132	132	132
query62	813	722	645	645
query63	224	185	185	185
query64	3482	1108	653	653
query65	3241	3187	3163	3163
query66	907	384	290	290
query67	16074	15749	15692	15692
query68	8233	827	522	522
query69	523	284	263	263
query70	1224	1088	1152	1088
query71	385	290	254	254
query72	5105	3879	3842	3842
query73	642	761	351	351
query74	10004	9154	8930	8930
query75	3191	3136	2663	2663
query76	3146	1153	759	759
query77	463	356	266	266
query78	10152	10346	9413	9413
query79	2775	831	603	603
query80	660	512	427	427
query81	504	272	243	243
query82	225	152	124	124
query83	168	164	151	151
query84	237	95	75	75
query85	734	344	288	288
query86	380	315	292	292
query87	4473	4512	4408	4408
query88	4518	2158	2113	2113
query89	395	320	307	307
query90	1894	188	183	183
query91	126	137	107	107
query92	67	57	52	52
query93	2226	858	523	523
query94	648	425	300	300
query95	339	274	256	256
query96	486	618	289	289
query97	2745	2851	2794	2794
query98	223	200	195	195
query99	1273	1356	1304	1304
Total cold run time: 296288 ms
Total hot run time: 193640 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6700cf92defd8af885db6045894a78250db1cd8c, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.04	0.04
query3	0.24	0.07	0.08
query4	1.62	0.11	0.10
query5	0.41	0.42	0.40
query6	1.16	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.59	0.50	0.51
query10	0.54	0.56	0.55
query11	0.14	0.10	0.11
query12	0.14	0.10	0.11
query13	0.60	0.61	0.59
query14	2.82	2.88	2.85
query15	0.92	0.82	0.84
query16	0.39	0.39	0.39
query17	1.07	1.01	1.06
query18	0.23	0.20	0.20
query19	1.97	1.83	1.94
query20	0.01	0.01	0.01
query21	15.37	0.94	0.57
query22	0.75	0.87	0.62
query23	15.25	1.45	0.51
query24	2.69	0.32	0.92
query25	0.26	0.16	0.14
query26	0.21	0.14	0.14
query27	0.04	0.06	0.05
query28	13.79	1.09	0.45
query29	12.55	3.95	3.30
query30	0.25	0.08	0.06
query31	2.84	0.59	0.37
query32	3.23	0.56	0.46
query33	3.06	3.06	2.99
query34	16.66	5.21	4.52
query35	4.52	4.46	4.53
query36	0.66	0.49	0.48
query37	0.10	0.06	0.07
query38	0.05	0.04	0.04
query39	0.04	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.65 s
Total hot run time: 30.05 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants