Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[test](index compaction) Add index compaction full flow UT test #45746

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

qidaye
Copy link
Contributor

@qidaye qidaye commented Dec 20, 2024

What problem does this PR solve?

  1. Add index compaction full flow UT tests
  2. Add index compaction performance test, disable by default.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 20, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@qidaye
Copy link
Contributor Author

qidaye commented Dec 20, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39683 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e53b0519d8c749cbe4f19a9dc0c5329e8893f858, data reload: false

------ Round 1 ----------------------------------
q1	17583	7545	7295	7295
q2	2052	183	177	177
q3	10584	1101	1133	1101
q4	10528	739	683	683
q5	7613	2687	2744	2687
q6	237	148	142	142
q7	976	618	609	609
q8	9256	1892	1932	1892
q9	6685	6358	6462	6358
q10	7043	2304	2298	2298
q11	478	257	255	255
q12	444	231	223	223
q13	17778	2996	2920	2920
q14	238	204	212	204
q15	574	529	503	503
q16	665	585	592	585
q17	984	561	545	545
q18	7340	6748	6645	6645
q19	1351	1027	955	955
q20	481	188	188	188
q21	4236	3180	3108	3108
q22	379	321	310	310
Total cold run time: 107505 ms
Total hot run time: 39683 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7236	7265	7269	7265
q2	331	228	231	228
q3	2895	2814	2954	2814
q4	2085	1810	1824	1810
q5	5708	5653	5664	5653
q6	222	143	145	143
q7	2290	1802	1823	1802
q8	3386	3580	3587	3580
q9	8895	9058	8970	8970
q10	3643	3546	3583	3546
q11	608	516	517	516
q12	832	652	584	584
q13	10821	3083	3044	3044
q14	310	294	271	271
q15	549	498	524	498
q16	696	655	651	651
q17	1854	1653	1629	1629
q18	8482	7879	7715	7715
q19	1788	1532	1623	1532
q20	2087	1905	1942	1905
q21	5637	5510	5646	5510
q22	641	587	586	586
Total cold run time: 70996 ms
Total hot run time: 60252 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196356 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e53b0519d8c749cbe4f19a9dc0c5329e8893f858, data reload: false

query1	1314	987	932	932
query2	6237	2525	2440	2440
query3	10947	4657	4713	4657
query4	33146	23468	23434	23434
query5	4092	482	485	482
query6	295	182	178	178
query7	3987	305	313	305
query8	294	229	223	223
query9	9564	2696	2683	2683
query10	471	247	251	247
query11	17874	15182	15211	15182
query12	157	108	102	102
query13	1590	446	413	413
query14	8761	6842	7606	6842
query15	261	199	192	192
query16	8173	451	505	451
query17	1621	634	625	625
query18	2221	341	336	336
query19	375	179	166	166
query20	124	124	127	124
query21	211	109	113	109
query22	4846	4571	4377	4377
query23	35560	33763	33582	33582
query24	10541	2535	2523	2523
query25	665	450	398	398
query26	1265	164	157	157
query27	2383	353	336	336
query28	8114	2463	2453	2453
query29	867	433	430	430
query30	239	149	145	145
query31	1057	817	830	817
query32	96	54	59	54
query33	742	331	284	284
query34	1009	524	524	524
query35	881	788	779	779
query36	1100	966	939	939
query37	152	77	75	75
query38	4455	4399	4236	4236
query39	1490	1472	1459	1459
query40	207	107	101	101
query41	46	42	45	42
query42	114	106	102	102
query43	573	494	493	493
query44	1311	866	839	839
query45	181	169	167	167
query46	1218	735	720	720
query47	2053	1916	1913	1913
query48	416	328	322	322
query49	876	387	390	387
query50	843	407	409	407
query51	7365	7242	7198	7198
query52	104	91	94	91
query53	274	182	185	182
query54	1162	433	433	433
query55	84	78	80	78
query56	276	252	255	252
query57	1308	1149	1126	1126
query58	236	229	229	229
query59	3382	3158	3151	3151
query60	288	258	255	255
query61	112	100	106	100
query62	883	708	680	680
query63	216	195	197	195
query64	3889	717	691	691
query65	3310	3270	3310	3270
query66	700	309	306	306
query67	16740	15529	15468	15468
query68	5315	550	546	546
query69	492	265	249	249
query70	1205	1097	1094	1094
query71	487	243	247	243
query72	7187	4166	4058	4058
query73	805	371	363	363
query74	10077	8837	8734	8734
query75	3554	2606	2637	2606
query76	3957	1137	1098	1098
query77	576	271	283	271
query78	10368	9458	9400	9400
query79	2743	611	625	611
query80	1369	423	442	423
query81	520	232	238	232
query82	695	121	130	121
query83	202	157	146	146
query84	286	68	72	68
query85	1324	312	299	299
query86	431	291	301	291
query87	4535	4278	4676	4278
query88	4506	2190	2198	2190
query89	445	283	296	283
query90	1901	190	192	190
query91	145	104	106	104
query92	66	53	50	50
query93	3900	550	535	535
query94	690	281	272	272
query95	360	251	253	251
query96	652	273	274	273
query97	2830	2688	2671	2671
query98	215	198	204	198
query99	1623	1349	1319	1319
Total cold run time: 308805 ms
Total hot run time: 196356 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.85 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e53b0519d8c749cbe4f19a9dc0c5329e8893f858, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.10	0.10
query5	0.42	0.42	0.39
query6	1.17	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.58	0.52	0.52
query10	0.56	0.58	0.55
query11	0.16	0.11	0.09
query12	0.14	0.12	0.10
query13	0.60	0.60	0.61
query14	2.86	2.87	2.74
query15	0.90	0.82	0.82
query16	0.39	0.38	0.37
query17	1.07	1.06	1.04
query18	0.23	0.21	0.21
query19	1.96	1.82	2.03
query20	0.02	0.01	0.01
query21	15.35	0.59	0.56
query22	3.04	1.89	1.78
query23	17.07	0.89	0.84
query24	3.05	1.37	1.35
query25	0.24	0.24	0.14
query26	0.48	0.14	0.14
query27	0.04	0.04	0.04
query28	9.94	1.10	1.07
query29	12.60	3.23	3.17
query30	0.24	0.06	0.05
query31	2.88	0.40	0.38
query32	3.24	0.46	0.46
query33	3.13	3.08	3.14
query34	16.92	4.46	4.44
query35	4.49	4.40	4.43
query36	0.69	0.47	0.50
query37	0.09	0.05	0.05
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.18	0.13	0.13
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.99 s
Total hot run time: 32.85 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants