Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix)(colocate join) fix wrong use of colocate join #37361

Merged
merged 4 commits into from
Jul 12, 2024

Conversation

cambyzju
Copy link
Contributor

@cambyzju cambyzju commented Jul 5, 2024

Proposed changes

Issue Number: close #36324

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@cambyzju
Copy link
Contributor Author

cambyzju commented Jul 5, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39824 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4fdf27de97b8661b00312f04825ec11c21b9c4aa, data reload: false

------ Round 1 ----------------------------------
q1	17627	4538	4309	4309
q2	2039	198	194	194
q3	10521	1271	1106	1106
q4	11227	721	774	721
q5	7614	2650	2672	2650
q6	228	142	141	141
q7	966	618	608	608
q8	9399	2118	2097	2097
q9	9011	6500	6534	6500
q10	8841	3737	3721	3721
q11	460	234	240	234
q12	555	243	227	227
q13	17772	2961	2982	2961
q14	278	224	214	214
q15	530	479	495	479
q16	528	393	382	382
q17	953	637	740	637
q18	8089	7444	7526	7444
q19	7093	1570	1363	1363
q20	698	309	325	309
q21	5015	3277	3203	3203
q22	380	324	336	324
Total cold run time: 119824 ms
Total hot run time: 39824 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4394	4290	4263	4263
q2	389	281	280	280
q3	3031	2754	2702	2702
q4	1857	1582	1590	1582
q5	5346	5259	5287	5259
q6	218	134	129	129
q7	2113	1730	1749	1730
q8	3170	3410	3280	3280
q9	8322	8317	8344	8317
q10	3872	3724	3618	3618
q11	577	492	485	485
q12	801	639	599	599
q13	16843	3023	2959	2959
q14	288	250	253	250
q15	514	485	484	484
q16	466	420	436	420
q17	1778	1500	1539	1500
q18	7688	7558	7476	7476
q19	1743	1581	1464	1464
q20	1986	1808	1756	1756
q21	9679	4763	4745	4745
q22	624	553	534	534
Total cold run time: 75699 ms
Total hot run time: 53832 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172986 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4fdf27de97b8661b00312f04825ec11c21b9c4aa, data reload: false

query1	929	379	373	373
query2	6456	2422	2286	2286
query3	6644	207	226	207
query4	23497	17407	17313	17313
query5	4191	500	494	494
query6	319	187	146	146
query7	4590	289	285	285
query8	335	280	284	280
query9	8434	2399	2404	2399
query10	602	297	278	278
query11	10686	10089	10181	10089
query12	133	92	85	85
query13	1628	389	385	385
query14	9334	7768	7585	7585
query15	253	183	185	183
query16	7593	309	300	300
query17	1825	548	521	521
query18	1875	282	281	281
query19	193	145	152	145
query20	89	79	82	79
query21	209	137	127	127
query22	4483	4408	4317	4317
query23	33634	33155	33119	33119
query24	11035	2898	2822	2822
query25	634	364	364	364
query26	1390	154	154	154
query27	2920	310	317	310
query28	7342	2060	2050	2050
query29	931	628	592	592
query30	283	147	145	145
query31	986	721	755	721
query32	93	57	57	57
query33	762	283	283	283
query34	987	488	486	486
query35	715	610	607	607
query36	1138	898	925	898
query37	151	79	75	75
query38	2894	2732	2779	2732
query39	865	784	797	784
query40	293	131	138	131
query41	59	59	53	53
query42	119	97	103	97
query43	571	540	514	514
query44	1185	728	737	728
query45	197	172	165	165
query46	1077	731	698	698
query47	1846	1805	1750	1750
query48	373	294	304	294
query49	1155	410	425	410
query50	772	386	388	386
query51	6998	6853	6769	6769
query52	99	98	92	92
query53	362	290	288	288
query54	1036	451	451	451
query55	74	75	73	73
query56	312	292	289	289
query57	1127	1052	1045	1045
query58	283	252	255	252
query59	3357	3106	3263	3106
query60	337	296	296	296
query61	117	118	115	115
query62	664	425	452	425
query63	325	293	301	293
query64	10691	2247	1702	1702
query65	3188	3095	3088	3088
query66	1404	352	342	342
query67	15606	15000	15016	15000
query68	6287	550	540	540
query69	714	432	350	350
query70	1187	1053	1155	1053
query71	460	277	274	274
query72	8197	5175	5707	5175
query73	762	319	316	316
query74	5883	5710	5528	5528
query75	4684	2799	2731	2731
query76	3820	957	860	860
query77	699	299	310	299
query78	9823	9005	9001	9001
query79	2935	518	508	508
query80	1379	476	488	476
query81	579	221	224	221
query82	1358	110	106	106
query83	301	167	169	167
query84	273	95	84	84
query85	1546	333	346	333
query86	471	322	304	304
query87	3328	3076	3105	3076
query88	4347	2339	2353	2339
query89	492	386	379	379
query90	1898	187	187	187
query91	135	100	100	100
query92	67	49	51	49
query93	3935	509	501	501
query94	1259	206	209	206
query95	401	311	301	301
query96	605	264	265	264
query97	3231	3069	3038	3038
query98	226	202	196	196
query99	1188	815	866	815
Total cold run time: 287155 ms
Total hot run time: 172986 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4fdf27de97b8661b00312f04825ec11c21b9c4aa, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.07	0.08
query5	0.51	0.50	0.48
query6	1.12	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.54	0.48	0.50
query10	0.54	0.53	0.56
query11	0.16	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.59	0.59
query14	0.78	0.78	0.78
query15	0.84	0.81	0.80
query16	0.35	0.38	0.37
query17	0.95	0.94	1.05
query18	0.22	0.24	0.24
query19	1.89	1.79	1.75
query20	0.01	0.01	0.00
query21	15.40	0.73	0.66
query22	4.25	6.53	1.93
query23	18.27	1.26	1.21
query24	2.13	0.24	0.22
query25	0.15	0.08	0.08
query26	0.28	0.21	0.21
query27	0.45	0.23	0.23
query28	13.21	1.02	1.00
query29	12.64	3.30	3.27
query30	0.25	0.06	0.05
query31	2.85	0.39	0.39
query32	3.29	0.47	0.46
query33	2.87	2.91	2.94
query34	16.99	4.35	4.34
query35	4.42	4.41	4.37
query36	0.64	0.47	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.08	0.05	0.04
query42	0.06	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.53 s
Total hot run time: 30.5 s

@cambyzju
Copy link
Contributor Author

cambyzju commented Jul 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39757 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a573581077a07713ae662fdcd6980bb5c2f22583, data reload: false

------ Round 1 ----------------------------------
q1	17638	4365	4311	4311
q2	2041	200	199	199
q3	10432	1146	1174	1146
q4	10203	770	725	725
q5	7513	2665	2590	2590
q6	222	142	142	142
q7	958	594	601	594
q8	9230	2058	2066	2058
q9	8823	6509	6496	6496
q10	9011	3730	3742	3730
q11	460	258	245	245
q12	538	230	228	228
q13	17864	2998	3003	2998
q14	267	216	238	216
q15	531	481	503	481
q16	516	394	380	380
q17	952	708	618	618
q18	7939	7473	7445	7445
q19	5581	1506	1477	1477
q20	650	311	330	311
q21	5013	3034	3877	3034
q22	397	333	350	333
Total cold run time: 116779 ms
Total hot run time: 39757 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4430	4256	4244	4244
q2	378	270	269	269
q3	2961	2846	2872	2846
q4	2003	1745	1707	1707
q5	5670	5482	5467	5467
q6	229	130	131	130
q7	2152	1873	1926	1873
q8	3232	3364	3370	3364
q9	8719	8682	8870	8682
q10	4064	3868	3691	3691
q11	591	509	497	497
q12	795	653	654	653
q13	16165	3192	3159	3159
q14	314	289	263	263
q15	541	500	493	493
q16	528	461	448	448
q17	1834	1540	1502	1502
q18	7989	7928	7935	7928
q19	1777	1707	1641	1641
q20	2146	1860	1849	1849
q21	5263	5040	4785	4785
q22	611	554	587	554
Total cold run time: 72392 ms
Total hot run time: 56045 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174074 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a573581077a07713ae662fdcd6980bb5c2f22583, data reload: false

query1	906	382	360	360
query2	6431	2511	2375	2375
query3	6632	201	213	201
query4	28291	17723	17385	17385
query5	3647	505	470	470
query6	268	179	175	175
query7	4599	298	290	290
query8	307	295	286	286
query9	8624	2375	2354	2354
query10	585	296	277	277
query11	11664	10223	10214	10214
query12	121	88	82	82
query13	1645	387	376	376
query14	10154	7598	7695	7598
query15	275	181	185	181
query16	7912	314	315	314
query17	1795	562	547	547
query18	1980	295	282	282
query19	207	155	155	155
query20	92	85	84	84
query21	212	135	129	129
query22	4520	4132	3932	3932
query23	34256	33760	33693	33693
query24	10857	2901	2822	2822
query25	587	425	419	419
query26	721	163	159	159
query27	2246	330	330	330
query28	6066	2130	2125	2125
query29	903	658	686	658
query30	248	155	155	155
query31	972	806	769	769
query32	102	53	57	53
query33	689	315	312	312
query34	974	508	502	502
query35	765	644	627	627
query36	1109	971	941	941
query37	136	77	81	77
query38	2922	2802	2779	2779
query39	871	795	802	795
query40	214	133	126	126
query41	52	50	49	49
query42	124	97	108	97
query43	606	565	548	548
query44	1083	728	733	728
query45	203	155	157	155
query46	1074	722	734	722
query47	1842	1781	1765	1765
query48	379	311	294	294
query49	852	413	411	411
query50	791	392	409	392
query51	6875	6743	6781	6743
query52	101	94	91	91
query53	356	284	291	284
query54	904	450	450	450
query55	74	74	70	70
query56	282	270	262	262
query57	1114	1050	1059	1050
query58	242	241	240	240
query59	3699	3136	3241	3136
query60	325	273	288	273
query61	94	92	90	90
query62	590	437	450	437
query63	315	292	289	289
query64	9162	2234	1619	1619
query65	3131	3105	3114	3105
query66	730	327	323	323
query67	15771	14853	14916	14853
query68	5134	523	530	523
query69	739	434	351	351
query70	1139	1150	1092	1092
query71	508	281	276	276
query72	8425	5725	5418	5418
query73	772	328	328	328
query74	6023	5607	5494	5494
query75	5102	2743	2755	2743
query76	4153	993	962	962
query77	740	296	290	290
query78	9606	8941	8882	8882
query79	7207	514	518	514
query80	1303	484	476	476
query81	584	222	226	222
query82	1531	106	108	106
query83	456	183	160	160
query84	276	88	89	88
query85	1419	308	308	308
query86	457	316	324	316
query87	3301	3103	3133	3103
query88	4987	2467	2477	2467
query89	489	397	382	382
query90	1944	187	192	187
query91	132	101	110	101
query92	60	48	47	47
query93	6153	501	492	492
query94	1283	207	214	207
query95	403	318	316	316
query96	604	275	272	272
query97	3217	3024	3020	3020
query98	220	198	194	194
query99	1136	852	903	852
Total cold run time: 295779 ms
Total hot run time: 174074 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.02 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a573581077a07713ae662fdcd6980bb5c2f22583, data reload: false

query1	0.04	0.04	0.04
query2	0.08	0.04	0.03
query3	0.23	0.05	0.05
query4	1.68	0.07	0.07
query5	0.51	0.48	0.49
query6	1.16	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.51	0.49
query10	0.54	0.54	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.13
query13	0.59	0.58	0.59
query14	0.77	0.77	0.76
query15	0.84	0.82	0.82
query16	0.36	0.36	0.37
query17	0.97	1.00	1.02
query18	0.25	0.22	0.26
query19	1.80	1.69	1.70
query20	0.01	0.01	0.00
query21	15.42	0.75	0.67
query22	4.45	6.75	2.20
query23	18.30	1.38	1.28
query24	2.16	0.22	0.22
query25	0.15	0.09	0.08
query26	0.28	0.20	0.21
query27	0.44	0.23	0.23
query28	13.27	1.01	1.00
query29	12.61	3.33	3.31
query30	0.25	0.06	0.05
query31	2.87	0.39	0.39
query32	3.26	0.48	0.47
query33	2.84	3.00	2.91
query34	17.06	4.40	4.39
query35	4.40	4.47	4.46
query36	0.66	0.47	0.48
query37	0.18	0.15	0.16
query38	0.16	0.14	0.13
query39	0.04	0.03	0.04
query40	0.15	0.12	0.13
query41	0.09	0.04	0.05
query42	0.06	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.9 s
Total hot run time: 31.02 s

@cambyzju
Copy link
Contributor Author

cambyzju commented Jul 8, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40107 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6d6d906cb3cd5da073b1fb1ccd2f4fb796f8726, data reload: false

------ Round 1 ----------------------------------
q1	17639	4335	4368	4335
q2	2016	200	189	189
q3	10567	1196	1167	1167
q4	10726	695	767	695
q5	7595	2695	2659	2659
q6	226	138	139	138
q7	973	615	602	602
q8	9610	2131	2122	2122
q9	8965	6636	6643	6636
q10	9135	3690	3707	3690
q11	469	241	245	241
q12	499	245	225	225
q13	17804	2972	3014	2972
q14	276	233	213	213
q15	536	491	490	490
q16	546	398	386	386
q17	983	681	682	681
q18	8095	7444	7391	7391
q19	3909	1490	1497	1490
q20	686	328	328	328
q21	5081	3125	3312	3125
q22	398	332	343	332
Total cold run time: 116734 ms
Total hot run time: 40107 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4395	4303	4315	4303
q2	381	281	276	276
q3	2966	2859	2858	2858
q4	1988	1694	1724	1694
q5	5495	5474	5534	5474
q6	235	135	135	135
q7	2270	1916	1873	1873
q8	3252	3385	3401	3385
q9	8609	8682	8709	8682
q10	4052	3887	3908	3887
q11	605	498	480	480
q12	824	652	642	642
q13	17850	3139	3236	3139
q14	309	288	281	281
q15	544	500	491	491
q16	507	476	442	442
q17	1842	1497	1499	1497
q18	8264	8064	7843	7843
q19	3989	1563	1554	1554
q20	2153	1861	1808	1808
q21	5144	4943	4866	4866
q22	635	555	554	554
Total cold run time: 76309 ms
Total hot run time: 56164 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174178 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6d6d906cb3cd5da073b1fb1ccd2f4fb796f8726, data reload: false

query1	919	370	354	354
query2	6193	2309	2240	2240
query3	6643	206	215	206
query4	28401	17488	17376	17376
query5	3927	485	479	479
query6	256	178	169	169
query7	4604	303	314	303
query8	325	301	295	295
query9	8501	2394	2376	2376
query10	579	300	306	300
query11	10642	9948	10058	9948
query12	137	88	83	83
query13	1654	376	382	376
query14	10173	7564	7514	7514
query15	235	187	186	186
query16	7742	298	311	298
query17	1820	548	541	541
query18	1773	285	276	276
query19	199	146	151	146
query20	92	80	81	80
query21	208	130	129	129
query22	4239	4096	4045	4045
query23	34017	33695	33881	33695
query24	10991	2775	2805	2775
query25	660	391	428	391
query26	1605	162	156	156
query27	3054	325	335	325
query28	7453	2169	2113	2113
query29	985	662	680	662
query30	255	158	151	151
query31	1007	752	768	752
query32	101	56	54	54
query33	752	301	309	301
query34	954	503	496	496
query35	748	676	644	644
query36	1153	974	960	960
query37	159	78	83	78
query38	2995	2838	2770	2770
query39	867	813	802	802
query40	268	125	130	125
query41	52	52	54	52
query42	122	98	100	98
query43	626	526	520	520
query44	1214	742	737	737
query45	198	159	163	159
query46	1082	728	719	719
query47	1863	1758	1746	1746
query48	389	306	302	302
query49	930	420	430	420
query50	759	381	385	381
query51	6860	6812	6779	6779
query52	107	89	94	89
query53	356	292	291	291
query54	833	461	455	455
query55	72	73	75	73
query56	298	266	269	266
query57	1181	1024	1021	1021
query58	284	257	247	247
query59	3475	3181	3107	3107
query60	302	284	283	283
query61	101	97	97	97
query62	595	440	438	438
query63	310	283	287	283
query64	10320	2163	7395	2163
query65	3147	3115	3127	3115
query66	1253	367	325	325
query67	15454	14937	15230	14937
query68	4516	538	549	538
query69	572	402	361	361
query70	1133	1107	1079	1079
query71	403	284	283	283
query72	6982	5299	5585	5299
query73	728	337	322	322
query74	6050	5511	5495	5495
query75	3495	2782	2796	2782
query76	2758	866	887	866
query77	435	307	324	307
query78	9505	9185	8938	8938
query79	2626	513	514	513
query80	2290	488	478	478
query81	597	221	214	214
query82	737	103	108	103
query83	301	178	172	172
query84	269	88	85	85
query85	2186	325	313	313
query86	487	314	311	311
query87	3342	3188	3101	3101
query88	3993	2465	2457	2457
query89	472	375	383	375
query90	1835	189	188	188
query91	135	102	101	101
query92	63	49	48	48
query93	1937	522	513	513
query94	1234	210	209	209
query95	399	318	310	310
query96	596	272	277	272
query97	3192	3023	3027	3023
query98	228	207	204	204
query99	1187	850	845	845
Total cold run time: 284152 ms
Total hot run time: 174178 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.64 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6d6d906cb3cd5da073b1fb1ccd2f4fb796f8726, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.05	0.05
query4	1.66	0.09	0.10
query5	0.51	0.49	0.50
query6	1.15	0.74	0.72
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.55	0.49	0.47
query10	0.54	0.53	0.56
query11	0.14	0.11	0.12
query12	0.14	0.12	0.13
query13	0.60	0.58	0.58
query14	0.76	0.78	0.78
query15	0.83	0.82	0.81
query16	0.36	0.35	0.37
query17	0.97	1.01	0.95
query18	0.21	0.28	0.22
query19	1.82	1.66	1.78
query20	0.03	0.01	0.00
query21	15.41	0.77	0.65
query22	3.95	6.86	2.11
query23	18.31	1.37	1.19
query24	2.06	0.24	0.23
query25	0.16	0.09	0.09
query26	0.29	0.21	0.21
query27	0.45	0.22	0.22
query28	13.23	1.02	1.00
query29	12.70	3.29	3.27
query30	0.25	0.06	0.05
query31	2.86	0.39	0.38
query32	3.29	0.47	0.46
query33	2.85	2.96	2.91
query34	17.14	4.33	4.34
query35	4.43	4.45	4.45
query36	0.65	0.46	0.49
query37	0.18	0.15	0.16
query38	0.15	0.14	0.14
query39	0.05	0.04	0.04
query40	0.15	0.12	0.12
query41	0.09	0.05	0.04
query42	0.06	0.05	0.06
query43	0.04	0.04	0.03
Total cold run time: 109.42 s
Total hot run time: 30.64 s

// on conditions must keep same order as distributed columns
Integer leftIndex = leftHashSpec.getExprIdToEquivalenceSet().get(((Slot) leftChild).getExprId());
Integer rightIndex = rightHashSpec.getExprIdToEquivalenceSet().get(((Slot) rightChild).getExprId());
if (leftIndex != rightIndex) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use Objects.equals to avoid npe when leftIndex is null

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got, I will fix it later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 8, 2024
Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by anyone and no changes requested.

1 similar comment
Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by anyone and no changes requested.

morrySnow
morrySnow previously approved these changes Jul 8, 2024
Comment on lines +318 to +319
Expression leftChild = ((EqualPredicate) expr).left();
Expression rightChild = ((EqualPredicate) expr).right();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not always left for left child, right for right child

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, maybe it is the reason why lots of regression tests of tcpds + tpch shape failed, I will try to fix it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@morrySnow morrySnow self-requested a review July 8, 2024 09:40
@cambyzju
Copy link
Contributor Author

cambyzju commented Jul 9, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40036 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dc4dfe8086fa434223ac5fdeee17732c92eb75ef, data reload: false

------ Round 1 ----------------------------------
q1	17598	4386	4263	4263
q2	2019	191	193	191
q3	10467	1200	1101	1101
q4	10203	841	723	723
q5	7503	2699	2719	2699
q6	224	139	137	137
q7	977	603	613	603
q8	9230	2087	2091	2087
q9	8930	6517	6505	6505
q10	9019	3807	3730	3730
q11	469	248	249	248
q12	484	233	243	233
q13	18763	2965	3006	2965
q14	271	227	234	227
q15	521	494	488	488
q16	524	383	380	380
q17	969	691	763	691
q18	8024	7503	7440	7440
q19	6636	1516	1473	1473
q20	669	339	354	339
q21	5022	3269	3176	3176
q22	398	337	337	337
Total cold run time: 118920 ms
Total hot run time: 40036 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4375	4209	4248	4209
q2	361	268	268	268
q3	3047	2901	2901	2901
q4	2016	1720	1744	1720
q5	5580	5579	5469	5469
q6	221	134	133	133
q7	2234	1884	1852	1852
q8	3342	3463	3422	3422
q9	8799	8791	8803	8791
q10	4128	3783	3886	3783
q11	597	518	501	501
q12	825	660	626	626
q13	17102	3173	3213	3173
q14	317	280	275	275
q15	532	471	492	471
q16	504	438	426	426
q17	1822	1483	1522	1483
q18	8197	7897	7833	7833
q19	1871	1559	1580	1559
q20	2987	1868	1865	1865
q21	5089	4883	4835	4835
q22	646	547	541	541
Total cold run time: 74592 ms
Total hot run time: 56136 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173973 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dc4dfe8086fa434223ac5fdeee17732c92eb75ef, data reload: false

query1	924	379	370	370
query2	6440	2305	2248	2248
query3	6635	209	213	209
query4	28185	17502	17305	17305
query5	3659	483	481	481
query6	280	167	179	167
query7	4583	296	273	273
query8	296	276	287	276
query9	8486	2453	2426	2426
query10	571	298	282	282
query11	10860	10027	9978	9978
query12	119	91	86	86
query13	1660	378	371	371
query14	10109	7650	7744	7650
query15	245	195	189	189
query16	7766	322	311	311
query17	1814	572	545	545
query18	1840	297	288	288
query19	205	153	158	153
query20	100	83	85	83
query21	214	136	128	128
query22	4342	4104	3989	3989
query23	33886	33653	33628	33628
query24	10916	2923	2876	2876
query25	626	416	440	416
query26	708	160	152	152
query27	2168	278	273	273
query28	5841	2122	2112	2112
query29	916	648	661	648
query30	257	160	159	159
query31	983	782	788	782
query32	97	57	57	57
query33	801	328	305	305
query34	984	493	498	493
query35	730	581	595	581
query36	1102	986	944	944
query37	143	81	87	81
query38	2937	2803	2776	2776
query39	905	857	818	818
query40	198	121	117	117
query41	49	45	51	45
query42	114	99	97	97
query43	568	553	543	543
query44	1227	726	723	723
query45	194	155	158	155
query46	1075	715	701	701
query47	1868	1763	1774	1763
query48	374	294	304	294
query49	837	443	437	437
query50	773	388	400	388
query51	6800	6685	6748	6685
query52	109	92	93	92
query53	366	283	289	283
query54	944	448	431	431
query55	80	74	75	74
query56	297	262	262	262
query57	1127	1044	1041	1041
query58	252	253	261	253
query59	3364	3144	2999	2999
query60	308	280	296	280
query61	97	95	95	95
query62	799	653	658	653
query63	322	297	295	295
query64	9118	2216	1629	1629
query65	3133	3163	3119	3119
query66	711	328	330	328
query67	15791	15151	14926	14926
query68	7325	521	538	521
query69	772	439	358	358
query70	1179	1114	1064	1064
query71	489	290	272	272
query72	8968	5463	5488	5463
query73	840	322	317	317
query74	5996	5609	5503	5503
query75	4349	2666	2678	2666
query76	4524	942	894	894
query77	783	303	306	303
query78	9801	8990	8856	8856
query79	8493	510	524	510
query80	1117	467	479	467
query81	586	224	222	222
query82	748	141	129	129
query83	309	168	170	168
query84	278	85	88	85
query85	1092	328	369	328
query86	381	333	326	326
query87	3300	3109	3076	3076
query88	4900	2367	2351	2351
query89	547	382	378	378
query90	2004	193	190	190
query91	134	104	107	104
query92	62	48	50	48
query93	7056	496	493	493
query94	1252	209	206	206
query95	414	316	311	311
query96	617	271	261	261
query97	3215	2996	3003	2996
query98	212	197	197	197
query99	1556	1293	1301	1293
Total cold run time: 297527 ms
Total hot run time: 173973 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.76 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dc4dfe8086fa434223ac5fdeee17732c92eb75ef, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.04	0.04
query3	0.22	0.04	0.05
query4	1.68	0.08	0.07
query5	0.51	0.48	0.48
query6	1.14	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.04	0.05
query9	0.56	0.49	0.50
query10	0.55	0.53	0.55
query11	0.15	0.11	0.12
query12	0.15	0.13	0.12
query13	0.59	0.58	0.58
query14	0.77	0.77	0.77
query15	0.85	0.81	0.80
query16	0.35	0.36	0.38
query17	1.00	1.04	0.97
query18	0.22	0.21	0.22
query19	1.76	1.70	1.74
query20	0.01	0.01	0.02
query21	15.39	0.76	0.66
query22	4.24	6.82	2.17
query23	18.28	1.42	1.34
query24	2.13	0.22	0.22
query25	0.16	0.08	0.09
query26	0.29	0.20	0.21
query27	0.45	0.22	0.23
query28	13.27	1.03	1.01
query29	12.61	3.31	3.24
query30	0.25	0.06	0.05
query31	2.86	0.38	0.39
query32	3.29	0.49	0.48
query33	2.93	2.85	2.85
query34	17.23	4.36	4.32
query35	4.38	4.42	4.42
query36	0.64	0.47	0.48
query37	0.18	0.15	0.16
query38	0.16	0.14	0.14
query39	0.04	0.03	0.04
query40	0.15	0.12	0.13
query41	0.09	0.04	0.04
query42	0.06	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.81 s
Total hot run time: 30.76 s

@morrySnow morrySnow added the p0_w label Jul 9, 2024
@cambyzju cambyzju merged commit 7216734 into apache:master Jul 12, 2024
29 of 34 checks passed
cambyzju added a commit to cambyzju/incubator-doris that referenced this pull request Jul 12, 2024
morrySnow pushed a commit that referenced this pull request Jul 15, 2024
cambyzju added a commit to cambyzju/incubator-doris that referenced this pull request Jul 16, 2024
morrySnow pushed a commit that referenced this pull request Jul 17, 2024
seawinde pushed a commit to seawinde/doris that referenced this pull request Jul 17, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] wrong colocate join used
6 participants