Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](hudi) remove session variable field in HudiScanNode #45762

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

morningman
Copy link
Contributor

@morningman morningman commented Dec 22, 2024

What problem does this PR solve?

Related PR: #45355

Problem Summary:
The sessionVariable field is already in parent class FileQueryScanNode,
remove it from HudiScanNode.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 22, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

run buildall

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39622 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5bc137554fa631b3f2cb0bd3821c085365265f37, data reload: false

------ Round 1 ----------------------------------
q1	17608	7340	7214	7214
q2	2050	180	173	173
q3	10587	1068	1142	1068
q4	10552	750	745	745
q5	7623	2714	2705	2705
q6	235	148	143	143
q7	990	623	605	605
q8	9247	1802	1901	1802
q9	6674	6479	6386	6386
q10	6984	2283	2319	2283
q11	470	253	252	252
q12	441	219	219	219
q13	17787	2885	2968	2885
q14	249	204	215	204
q15	550	508	501	501
q16	671	583	581	581
q17	987	547	514	514
q18	7305	6717	6651	6651
q19	1343	1004	945	945
q20	479	180	185	180
q21	4020	3257	3289	3257
q22	384	324	309	309
Total cold run time: 107236 ms
Total hot run time: 39622 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7216	7239	7241	7239
q2	327	233	232	232
q3	2910	2748	2933	2748
q4	2078	1844	1783	1783
q5	5681	5657	5621	5621
q6	223	147	152	147
q7	2220	1801	1809	1801
q8	3332	3548	3491	3491
q9	8999	8975	8942	8942
q10	3567	3530	3543	3530
q11	600	507	502	502
q12	859	651	630	630
q13	13191	3038	3172	3038
q14	307	273	271	271
q15	567	504	511	504
q16	700	632	644	632
q17	1847	1612	1588	1588
q18	8243	7781	7768	7768
q19	1720	1580	1518	1518
q20	2013	1903	1862	1862
q21	5627	5484	5508	5484
q22	632	594	611	594
Total cold run time: 72859 ms
Total hot run time: 59925 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196440 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5bc137554fa631b3f2cb0bd3821c085365265f37, data reload: false

query1	1270	932	930	930
query2	6230	2317	2278	2278
query3	11127	4884	4850	4850
query4	33192	23446	23520	23446
query5	4349	477	452	452
query6	295	186	183	183
query7	3993	307	310	307
query8	313	245	234	234
query9	9392	2683	2668	2668
query10	475	255	247	247
query11	17909	15274	15108	15108
query12	163	105	104	104
query13	1582	436	416	416
query14	10376	7071	7275	7071
query15	294	187	185	185
query16	7980	479	487	479
query17	1752	593	561	561
query18	2129	304	306	304
query19	358	167	162	162
query20	122	110	114	110
query21	209	107	102	102
query22	4865	4799	4831	4799
query23	35331	33940	33639	33639
query24	10205	2514	2366	2366
query25	601	400	405	400
query26	1240	149	155	149
query27	2312	338	337	337
query28	7455	2415	2427	2415
query29	846	407	420	407
query30	234	149	156	149
query31	1027	863	848	848
query32	100	61	60	60
query33	778	310	315	310
query34	1102	513	554	513
query35	881	774	752	752
query36	1122	956	950	950
query37	129	75	76	75
query38	4321	4227	4144	4144
query39	1529	1443	1467	1443
query40	202	102	138	102
query41	44	43	44	43
query42	118	105	103	103
query43	542	509	492	492
query44	1353	839	841	839
query45	196	171	168	168
query46	1198	749	745	745
query47	2071	1932	1940	1932
query48	411	316	331	316
query49	903	416	439	416
query50	836	400	387	387
query51	7434	7218	7195	7195
query52	102	92	91	91
query53	257	183	183	183
query54	1193	408	407	407
query55	87	81	87	81
query56	277	243	231	231
query57	1306	1191	1159	1159
query58	245	225	240	225
query59	3290	3098	2945	2945
query60	279	257	249	249
query61	109	149	109	109
query62	839	711	678	678
query63	228	192	202	192
query64	3950	670	644	644
query65	3323	3275	3233	3233
query66	704	298	299	298
query67	16645	15649	15517	15517
query68	4653	550	549	549
query69	442	254	253	253
query70	1162	1108	1114	1108
query71	411	261	258	258
query72	6441	4009	4090	4009
query73	777	362	361	361
query74	9704	8916	8809	8809
query75	3392	2620	2661	2620
query76	2981	1095	1161	1095
query77	557	265	291	265
query78	10226	9349	9379	9349
query79	1129	580	601	580
query80	639	439	434	434
query81	510	236	229	229
query82	276	121	119	119
query83	164	144	146	144
query84	235	69	69	69
query85	947	307	371	307
query86	335	295	298	295
query87	4451	4513	4332	4332
query88	3623	2227	2189	2189
query89	422	293	299	293
query90	2013	189	183	183
query91	133	101	111	101
query92	62	49	49	49
query93	1419	538	527	527
query94	704	253	284	253
query95	349	252	255	252
query96	623	284	281	281
query97	2830	2699	2713	2699
query98	225	195	200	195
query99	1552	1326	1300	1300
Total cold run time: 299117 ms
Total hot run time: 196440 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.49 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5bc137554fa631b3f2cb0bd3821c085365265f37, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.23	0.08	0.08
query4	1.59	0.11	0.10
query5	0.43	0.41	0.43
query6	1.17	0.64	0.65
query7	0.03	0.02	0.02
query8	0.05	0.03	0.04
query9	0.58	0.51	0.51
query10	0.56	0.56	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.10
query13	0.61	0.61	0.59
query14	2.74	2.77	2.84
query15	0.89	0.82	0.82
query16	0.37	0.39	0.40
query17	1.05	0.98	1.06
query18	0.22	0.21	0.20
query19	1.96	1.87	2.03
query20	0.01	0.01	0.02
query21	15.36	0.58	0.58
query22	2.97	1.67	1.64
query23	17.10	0.88	0.78
query24	3.11	0.99	1.03
query25	0.22	0.17	0.09
query26	0.41	0.13	0.14
query27	0.03	0.04	0.05
query28	10.61	1.10	1.07
query29	12.58	3.22	3.21
query30	0.25	0.06	0.07
query31	2.86	0.38	0.39
query32	3.25	0.47	0.46
query33	3.08	3.05	3.08
query34	16.95	4.51	4.51
query35	4.53	4.46	4.49
query36	0.69	0.48	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 107.38 s
Total hot run time: 32.49 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants