Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[improvement](statistics)Support drop cached stats. #39367

Merged
merged 1 commit into from
Aug 15, 2024
Merged

[improvement](statistics)Support drop cached stats. #39367

merged 1 commit into from
Aug 15, 2024

Conversation

Jibing-Li
Copy link
Contributor

Support drop cached stats. Usage:
drop cached stats table_name

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added the doing label Aug 14, 2024
@Jibing-Li Jibing-Li marked this pull request as ready for review August 14, 2024 11:39
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 37406 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1fe71311d8df38e518789fe87fa9646c15666cad, data reload: false

------ Round 1 ----------------------------------
q1	17601	4349	4338	4338
q2	2035	178	173	173
q3	11576	991	1167	991
q4	10515	682	732	682
q5	7754	2811	2817	2811
q6	227	134	134	134
q7	963	590	582	582
q8	9516	2015	2048	2015
q9	8023	6489	6484	6484
q10	7004	2192	2151	2151
q11	455	244	235	235
q12	397	216	214	214
q13	17755	2976	2984	2976
q14	278	239	240	239
q15	529	489	496	489
q16	494	398	390	390
q17	958	730	726	726
q18	7284	6771	6700	6700
q19	7408	1110	1022	1022
q20	696	325	341	325
q21	4214	2739	2832	2739
q22	1070	1009	990	990
Total cold run time: 116752 ms
Total hot run time: 37406 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4459	4264	4306	4264
q2	380	284	267	267
q3	2856	2625	2590	2590
q4	1922	1707	1695	1695
q5	5747	5777	5582	5582
q6	227	132	137	132
q7	2154	1697	1729	1697
q8	3293	3500	3405	3405
q9	8687	8648	8740	8648
q10	3568	3230	3271	3230
q11	599	499	516	499
q12	845	606	631	606
q13	17123	3187	3181	3181
q14	312	288	296	288
q15	523	498	498	498
q16	505	447	433	433
q17	1822	1566	1519	1519
q18	8069	7983	7804	7804
q19	1790	1539	1686	1539
q20	2096	1877	1938	1877
q21	5511	5256	5269	5256
q22	1129	1021	1041	1021
Total cold run time: 73617 ms
Total hot run time: 56031 ms

@Jibing-Li
Copy link
Contributor Author

run performance

@Jibing-Li
Copy link
Contributor Author

run cloud_p1

@doris-robot
Copy link

TPC-H: Total hot run time: 37549 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1fe71311d8df38e518789fe87fa9646c15666cad, data reload: false

------ Round 1 ----------------------------------
q1	17609	4367	4422	4367
q2	2031	178	177	177
q3	11741	1012	1119	1012
q4	10498	728	781	728
q5	7752	2774	2819	2774
q6	222	137	137	137
q7	949	587	602	587
q8	9526	2037	2077	2037
q9	8724	6538	6569	6538
q10	7049	2159	2255	2159
q11	485	239	245	239
q12	392	219	214	214
q13	17877	2958	2993	2958
q14	290	242	230	230
q15	537	481	489	481
q16	498	392	379	379
q17	974	717	691	691
q18	7435	6779	6686	6686
q19	6317	1059	1003	1003
q20	703	343	328	328
q21	3861	2820	2854	2820
q22	1074	1004	1017	1004
Total cold run time: 116544 ms
Total hot run time: 37549 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4509	4318	4234	4234
q2	382	277	259	259
q3	2815	2632	2602	2602
q4	1919	1676	1698	1676
q5	5649	5795	5591	5591
q6	221	136	133	133
q7	2137	1720	1768	1720
q8	3344	3537	3430	3430
q9	8770	8767	8757	8757
q10	3584	3254	3301	3254
q11	614	519	523	519
q12	798	613	640	613
q13	15847	3114	3235	3114
q14	327	303	303	303
q15	544	498	488	488
q16	493	451	453	451
q17	1859	1543	1484	1484
q18	8075	7988	7812	7812
q19	1771	1610	1562	1562
q20	2181	1908	1900	1900
q21	9065	5460	5478	5460
q22	1134	1014	1007	1007
Total cold run time: 76038 ms
Total hot run time: 56369 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189368 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1fe71311d8df38e518789fe87fa9646c15666cad, data reload: false

query1	1258	873	867	867
query2	6445	1905	1880	1880
query3	10649	3888	3745	3745
query4	58227	24068	23473	23473
query5	5677	494	489	489
query6	476	169	167	167
query7	6395	289	287	287
query8	300	200	208	200
query9	8670	2446	2423	2423
query10	502	269	259	259
query11	17632	15126	14965	14965
query12	158	101	97	97
query13	1577	368	363	363
query14	12052	7268	7205	7205
query15	298	175	171	171
query16	7394	485	505	485
query17	1271	587	552	552
query18	1332	305	290	290
query19	258	150	151	150
query20	118	113	115	113
query21	206	108	99	99
query22	4440	4174	4459	4174
query23	34062	33803	32996	32996
query24	5666	2891	2882	2882
query25	521	381	370	370
query26	677	151	149	149
query27	1801	268	268	268
query28	3677	2076	2057	2057
query29	628	398	402	398
query30	350	151	147	147
query31	938	760	754	754
query32	81	55	55	55
query33	424	267	280	267
query34	844	448	483	448
query35	814	725	731	725
query36	1062	926	904	904
query37	135	79	79	79
query38	3922	3860	3835	3835
query39	1449	1421	1382	1382
query40	215	120	114	114
query41	47	46	46	46
query42	123	101	98	98
query43	495	474	453	453
query44	1058	732	732	732
query45	197	165	164	164
query46	1097	709	762	709
query47	1865	1756	1745	1745
query48	374	292	300	292
query49	786	428	467	428
query50	794	392	403	392
query51	6828	6727	6605	6605
query52	101	88	91	88
query53	261	177	185	177
query54	545	446	442	442
query55	75	73	74	73
query56	272	244	249	244
query57	1144	1068	1019	1019
query58	223	226	227	226
query59	2888	2725	2919	2725
query60	289	255	270	255
query61	94	94	99	94
query62	727	636	640	636
query63	206	180	182	180
query64	3237	1748	1703	1703
query65	3204	3133	3206	3133
query66	675	336	327	327
query67	15178	14717	14879	14717
query68	4521	530	549	530
query69	417	263	273	263
query70	1142	1080	1118	1080
query71	402	272	278	272
query72	6433	2284	2000	2000
query73	753	322	319	319
query74	9100	8795	8817	8795
query75	3383	2743	2677	2677
query76	2472	1030	960	960
query77	590	322	309	309
query78	9497	9125	9030	9030
query79	2301	564	556	556
query80	2628	469	492	469
query81	565	223	222	222
query82	778	138	131	131
query83	286	147	147	147
query84	285	82	76	76
query85	1230	281	275	275
query86	469	302	260	260
query87	4404	4244	4219	4219
query88	4059	2300	2302	2300
query89	397	277	285	277
query90	1917	194	194	194
query91	122	97	98	97
query92	61	52	51	51
query93	3044	544	526	526
query94	958	299	292	292
query95	358	269	268	268
query96	590	280	269	269
query97	3282	3041	3053	3041
query98	289	206	205	205
query99	1576	1271	1277	1271
Total cold run time: 313693 ms
Total hot run time: 189368 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.42 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1fe71311d8df38e518789fe87fa9646c15666cad, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.04	0.04
query3	0.23	0.04	0.05
query4	1.68	0.08	0.07
query5	0.48	0.50	0.49
query6	1.12	0.74	0.73
query7	0.02	0.01	0.01
query8	0.05	0.05	0.04
query9	0.54	0.48	0.47
query10	0.54	0.54	0.54
query11	0.15	0.11	0.11
query12	0.15	0.12	0.12
query13	0.60	0.61	0.60
query14	0.77	0.79	0.78
query15	0.88	0.81	0.82
query16	0.38	0.38	0.38
query17	0.98	1.04	1.04
query18	0.23	0.22	0.22
query19	1.80	1.66	1.83
query20	0.02	0.01	0.01
query21	15.38	0.73	0.66
query22	3.53	8.49	1.69
query23	18.30	1.38	1.24
query24	2.10	0.22	0.21
query25	0.14	0.07	0.08
query26	0.29	0.22	0.21
query27	0.45	0.23	0.22
query28	13.26	1.03	1.01
query29	12.60	3.33	3.25
query30	0.24	0.06	0.05
query31	2.89	0.39	0.39
query32	3.27	0.50	0.47
query33	2.94	3.01	2.98
query34	17.00	4.32	4.42
query35	4.44	4.42	4.40
query36	0.65	0.48	0.47
query37	0.18	0.16	0.16
query38	0.15	0.14	0.15
query39	0.04	0.04	0.03
query40	0.16	0.13	0.13
query41	0.09	0.04	0.04
query42	0.06	0.04	0.04
query43	0.05	0.04	0.04
Total cold run time: 108.95 s
Total hot run time: 30.42 s

morningman
morningman approved these changes Aug 15, 2024
View reviewed changes
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 15, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@Jibing-Li Jibing-Li merged commit 982dcf4 into apache:master Aug 15, 2024
30 of 31 checks passed
@Jibing-Li Jibing-Li deleted the refresh branch August 15, 2024 15:12
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 16, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (apache#39367)
7959949 
Support drop cached stats. Usage:
`drop cached stats table_name`
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 16, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (apache#39367)
56387d3 
Support drop cached stats. Usage:
`drop cached stats table_name`
@Jibing-Li Jibing-Li mentioned this pull request Aug 16, 2024
Merged
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 16, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (apache#39367)
8e30fee 
Support drop cached stats. Usage:
`drop cached stats table_name`
@Jibing-Li Jibing-Li mentioned this pull request Aug 16, 2024
Merged
Jibing-Li added a commit that referenced this pull request Aug 16, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (#39367) (#39462)
6aafc5a 
backport: #39367
dataroaring pushed a commit that referenced this pull request Aug 17, 2024
@Jibing-Li @dataroaring
[improvement](statistics)Support drop cached stats. (#39367)
c9ab71a 
Support drop cached stats. Usage:
`drop cached stats table_name`
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 19, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (apache#39367)
13ec130 
Support drop cached stats. Usage:
`drop cached stats table_name`
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 19, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (apache#39367)
7540557 
Support drop cached stats. Usage:
`drop cached stats table_name`
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 19, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (apache#39367)
5a12d7a 
Support drop cached stats. Usage:
`drop cached stats table_name`
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Aug 19, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (apache#39367)
7486df0 
Support drop cached stats. Usage:
`drop cached stats table_name`
Jibing-Li added a commit that referenced this pull request Aug 19, 2024
@Jibing-Li
[improvement](statistics)Support drop cached stats. (#39367) (#39461)
3cfe75e 
backport: #39367
@xiaokang xiaokang mentioned this pull request Sep 14, 2024
Open
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@Jibing-Li @Baymine
[improvement](statistics)Support drop cached stats. (apache#39367) (a…
79354d0 
…pache#39461)

backport: apache#39367
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morningman morningman morningman approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.15-merged dev/2.1.6-merged dev/3.0.2-merged doing reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Jibing-Li @doris-robot @morningman @morrySnow @dataroaring