Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[opt](nereids) optimize rewrite of synchronize materialize view (#45748) #45816

Merged

Conversation

924060929
Copy link
Contributor

cherry pick from #45748

…he#45748)

optimize rewrite of synchronize materialize view
1. cache toSql
2. fast parse UnboundSlot in NereidsParser.parseExpression

(cherry picked from commit e7d2fed)
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929 924060929 changed the base branch from master to branch-3.0 December 23, 2024 10:53
@924060929
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40425 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3bbe497057d55b2df39a2ad439c22766e4e3ab09, data reload: false

------ Round 1 ----------------------------------
q1	17566	7354	7219	7219
q2	2068	170	155	155
q3	10859	1068	1142	1068
q4	10551	780	784	780
q5	7722	2784	2780	2780
q6	236	148	143	143
q7	958	612	599	599
q8	9578	1906	1949	1906
q9	7993	6357	6472	6357
q10	7041	2334	2333	2333
q11	461	271	255	255
q12	409	221	215	215
q13	17921	2981	2962	2962
q14	246	206	204	204
q15	558	525	520	520
q16	685	601	586	586
q17	987	556	569	556
q18	7226	6516	6525	6516
q19	2577	995	1065	995
q20	467	204	189	189
q21	3986	3261	3135	3135
q22	1067	952	955	952
Total cold run time: 111162 ms
Total hot run time: 40425 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7325	7167	7241	7167
q2	320	230	242	230
q3	2978	2896	2844	2844
q4	2002	1859	1770	1770
q5	5684	5686	5696	5686
q6	221	146	151	146
q7	2129	1791	1773	1773
q8	3269	3504	3431	3431
q9	8783	8847	8779	8779
q10	3550	3528	3480	3480
q11	600	506	500	500
q12	801	603	575	575
q13	16411	3090	3112	3090
q14	296	270	276	270
q15	561	522	519	519
q16	695	658	682	658
q17	1843	1612	1611	1611
q18	8243	7863	7562	7562
q19	7414	1577	1525	1525
q20	2079	1865	1826	1826
q21	5406	5200	5147	5147
q22	1144	970	1029	970
Total cold run time: 81754 ms
Total hot run time: 59559 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193741 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3bbe497057d55b2df39a2ad439c22766e4e3ab09, data reload: false

query1	1290	937	919	919
query2	6233	2081	2007	2007
query3	10824	4085	3876	3876
query4	66123	29178	23366	23366
query5	5463	442	435	435
query6	435	174	175	174
query7	5665	308	313	308
query8	314	232	217	217
query9	9376	2658	2645	2645
query10	521	262	245	245
query11	17934	15138	15715	15138
query12	162	107	106	106
query13	1526	425	415	415
query14	10588	6529	6942	6529
query15	216	182	179	179
query16	7289	540	502	502
query17	1204	577	608	577
query18	1893	326	318	318
query19	219	154	151	151
query20	117	113	120	113
query21	68	59	44	44
query22	4508	4653	4297	4297
query23	34702	33729	33925	33729
query24	6067	2937	2843	2843
query25	504	383	388	383
query26	720	163	162	162
query27	1883	289	299	289
query28	4554	2567	2522	2522
query29	682	430	425	425
query30	244	164	162	162
query31	1000	815	827	815
query32	68	55	56	55
query33	473	285	281	281
query34	916	492	491	491
query35	855	748	742	742
query36	1105	942	927	927
query37	114	72	73	72
query38	4100	3969	4045	3969
query39	1490	1490	1467	1467
query40	135	83	85	83
query41	52	49	50	49
query42	116	98	103	98
query43	543	483	484	483
query44	1206	796	792	792
query45	188	171	170	170
query46	1163	718	719	718
query47	1998	1900	1897	1897
query48	460	374	378	374
query49	740	396	395	395
query50	836	418	422	418
query51	7245	7076	6984	6984
query52	110	86	91	86
query53	257	178	180	178
query54	565	448	451	448
query55	75	77	78	77
query56	266	247	244	244
query57	1194	1091	1131	1091
query58	214	211	215	211
query59	3169	2832	2978	2832
query60	282	262	259	259
query61	130	131	132	131
query62	772	656	650	650
query63	209	187	189	187
query64	1875	769	605	605
query65	3217	3144	3185	3144
query66	714	297	295	295
query67	15607	15646	15168	15168
query68	4584	542	538	538
query69	415	259	253	253
query70	1090	1098	1127	1098
query71	391	246	250	246
query72	6536	3921	3879	3879
query73	769	357	336	336
query74	10197	8811	8907	8811
query75	3299	2602	2659	2602
query76	2224	1136	1045	1045
query77	508	262	260	260
query78	10799	9742	9518	9518
query79	6531	602	577	577
query80	1809	431	425	425
query81	545	243	240	240
query82	957	115	108	108
query83	260	149	147	147
query84	296	83	79	79
query85	1672	311	273	273
query86	455	293	288	288
query87	4518	4226	4293	4226
query88	5523	2329	2360	2329
query89	411	291	290	290
query90	2097	177	180	177
query91	179	143	142	142
query92	66	48	50	48
query93	5765	523	522	522
query94	871	267	284	267
query95	337	246	244	244
query96	617	273	269	269
query97	3305	3060	3120	3060
query98	214	196	199	196
query99	1876	1286	1300	1286
Total cold run time: 333899 ms
Total hot run time: 193741 ms

@morrySnow morrySnow closed this Dec 24, 2024
@morrySnow morrySnow reopened this Dec 24, 2024
@morrySnow morrySnow merged commit 4fb8403 into apache:branch-3.0 Dec 24, 2024
23 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants