Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[refactor](sync job) disable sync job by default #45765

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

morningman
Copy link
Contributor

@morningman morningman commented Dec 22, 2024

What problem does this PR solve?

Problem Summary:

create sync job feature is no longer maintained, deprecated it by default and remove it in version 3.1.
And a new FE config enable_feature_data_sync_job

Release note

[refactor](sync job) disable sync job by default

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman
Copy link
Contributor Author

run buildall

HappenLee
HappenLee previously approved these changes Dec 22, 2024
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 22, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39656 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ee90d7c1d7492611bcfbe6dba6b2f8af40d09180, data reload: false

------ Round 1 ----------------------------------
q1	17577	7431	7253	7253
q2	2046	175	166	166
q3	10631	1097	1192	1097
q4	10577	738	856	738
q5	7600	2682	2675	2675
q6	239	148	150	148
q7	979	644	613	613
q8	9322	1858	1912	1858
q9	6634	6395	6461	6395
q10	7050	2374	2322	2322
q11	455	269	256	256
q12	431	229	222	222
q13	17759	2943	2931	2931
q14	240	223	215	215
q15	557	511	496	496
q16	662	601	581	581
q17	979	527	533	527
q18	7191	6662	6708	6662
q19	1357	1024	941	941
q20	470	189	183	183
q21	4063	3286	3059	3059
q22	381	319	318	318
Total cold run time: 107200 ms
Total hot run time: 39656 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7234	7212	7195	7195
q2	330	228	233	228
q3	2882	2798	2947	2798
q4	2066	1838	1828	1828
q5	5673	5651	5651	5651
q6	224	150	145	145
q7	2281	1811	1812	1811
q8	3364	3536	3474	3474
q9	8960	8988	8986	8986
q10	3572	3535	3528	3528
q11	602	500	502	500
q12	820	587	608	587
q13	12809	3144	3166	3144
q14	332	275	274	274
q15	547	519	501	501
q16	710	642	653	642
q17	1881	1614	1628	1614
q18	8300	7742	7779	7742
q19	1766	1481	1574	1481
q20	2053	1894	1841	1841
q21	5673	5628	5490	5490
q22	679	577	583	577
Total cold run time: 72758 ms
Total hot run time: 60037 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196416 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ee90d7c1d7492611bcfbe6dba6b2f8af40d09180, data reload: false

query1	1310	978	939	939
query2	6230	2401	2240	2240
query3	11116	4833	4630	4630
query4	33017	23373	23399	23373
query5	4114	467	453	453
query6	298	191	196	191
query7	4003	304	315	304
query8	308	246	244	244
query9	9596	2704	2697	2697
query10	462	272	250	250
query11	17964	15245	15098	15098
query12	157	104	106	104
query13	1585	450	423	423
query14	10412	7259	7818	7259
query15	293	191	182	182
query16	7941	466	471	466
query17	1767	599	576	576
query18	2005	302	312	302
query19	363	161	160	160
query20	120	119	114	114
query21	205	106	118	106
query22	4923	4443	4448	4443
query23	34878	33753	34734	33753
query24	11226	2597	2447	2447
query25	527	383	407	383
query26	747	146	152	146
query27	2518	334	333	333
query28	7501	2424	2462	2424
query29	670	410	426	410
query30	226	154	153	153
query31	1065	849	833	833
query32	105	57	59	57
query33	760	316	294	294
query34	957	523	533	523
query35	895	778	805	778
query36	1111	945	975	945
query37	120	77	77	77
query38	4315	4180	4108	4108
query39	1509	1461	1471	1461
query40	208	101	102	101
query41	50	45	44	44
query42	120	100	104	100
query43	540	504	489	489
query44	1345	833	833	833
query45	194	181	186	181
query46	1192	735	732	732
query47	2064	1955	1937	1937
query48	435	333	333	333
query49	892	411	396	396
query50	839	393	406	393
query51	7323	7212	7247	7212
query52	103	92	97	92
query53	265	178	186	178
query54	1185	447	417	417
query55	89	79	85	79
query56	268	249	244	244
query57	1268	1176	1160	1160
query58	247	238	227	227
query59	3344	3099	3229	3099
query60	269	268	257	257
query61	108	114	107	107
query62	878	690	676	676
query63	222	193	194	193
query64	3773	701	695	695
query65	3298	3227	3373	3227
query66	804	327	303	303
query67	16385	15491	15537	15491
query68	4999	556	578	556
query69	485	257	257	257
query70	1146	1136	1116	1116
query71	445	247	244	244
query72	6349	3844	4048	3844
query73	779	368	368	368
query74	10230	8810	9057	8810
query75	3437	2600	2657	2600
query76	3477	1066	1155	1066
query77	640	281	290	281
query78	10130	9494	9466	9466
query79	1891	608	607	607
query80	767	425	431	425
query81	512	229	228	228
query82	674	120	118	118
query83	171	145	140	140
query84	253	71	70	70
query85	1418	380	373	373
query86	517	304	305	304
query87	4557	4304	4542	4304
query88	4011	2210	2195	2195
query89	444	296	278	278
query90	1986	194	189	189
query91	150	102	106	102
query92	64	52	50	50
query93	2143	554	551	551
query94	795	280	293	280
query95	377	256	244	244
query96	628	277	290	277
query97	2853	2731	2671	2671
query98	220	203	204	203
query99	1589	1324	1334	1324
Total cold run time: 303164 ms
Total hot run time: 196416 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.7 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ee90d7c1d7492611bcfbe6dba6b2f8af40d09180, data reload: false

query1	0.03	0.03	0.05
query2	0.07	0.03	0.04
query3	0.24	0.08	0.08
query4	1.59	0.11	0.10
query5	0.41	0.41	0.40
query6	1.16	0.66	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.50	0.50
query10	0.56	0.55	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.61	0.60
query14	2.72	2.74	2.75
query15	0.89	0.83	0.82
query16	0.37	0.41	0.38
query17	1.06	1.01	1.08
query18	0.22	0.22	0.21
query19	1.93	1.78	1.95
query20	0.01	0.01	0.01
query21	15.36	0.60	0.58
query22	2.62	1.85	1.34
query23	16.96	1.08	0.81
query24	3.27	1.38	1.53
query25	0.31	0.19	0.14
query26	0.39	0.14	0.14
query27	0.05	0.04	0.04
query28	9.60	1.13	1.09
query29	12.57	3.23	3.18
query30	0.25	0.07	0.06
query31	2.84	0.40	0.39
query32	3.23	0.46	0.46
query33	2.99	3.21	3.19
query34	16.77	4.48	4.54
query35	4.56	4.47	4.48
query36	0.65	0.48	0.47
query37	0.10	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.02
query40	0.19	0.13	0.14
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.04	0.04
Total cold run time: 105.72 s
Total hot run time: 32.7 s

@morningman
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants