Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature](vault) Support alter storage vault stmt in FE #37606

Merged
merged 1 commit into from
Jul 29, 2024
Merged

[feature](vault) Support alter storage vault stmt in FE #37606

merged 1 commit into from
Jul 29, 2024

Conversation

ByteYue
Copy link
Contributor

@ByteYue ByteYue commented Jul 10, 2024 •
edited by gavinchou
Loading

Proposed changes

Support alter storage vault stmt in FE. Currently only support alter s3 vault's ak and sk.
For example

alter storage vault s3_vault properties(
"type" = "s3",
"ak" = "new_ak",
"sk" = "new_sk"
);

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@ByteYue
Copy link
Contributor Author

ByteYue commented Jul 10, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39857 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 283ca1f817cba2b71811203d35d808144ca12b43, data reload: false

------ Round 1 ----------------------------------
q1	17626	4306	4238	4238
q2	2005	194	182	182
q3	10671	1173	1135	1135
q4	10674	726	733	726
q5	8562	2689	2684	2684
q6	229	141	140	140
q7	984	615	625	615
q8	10024	2075	2087	2075
q9	9262	6593	6504	6504
q10	8911	3698	3697	3697
q11	447	235	242	235
q12	393	228	230	228
q13	17766	2953	2973	2953
q14	271	224	219	219
q15	525	472	477	472
q16	457	380	370	370
q17	954	610	666	610
q18	7994	7567	7544	7544
q19	2981	1499	1401	1401
q20	687	321	319	319
q21	5243	3172	3908	3172
q22	399	348	338	338
Total cold run time: 117065 ms
Total hot run time: 39857 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4363	4222	4259	4222
q2	374	266	260	260
q3	3032	2902	2898	2898
q4	1995	1712	1722	1712
q5	5543	5480	5424	5424
q6	219	128	135	128
q7	2242	1788	1887	1788
q8	3294	3425	3394	3394
q9	8702	8492	8686	8492
q10	4055	3819	3812	3812
q11	587	484	494	484
q12	800	654	654	654
q13	17668	3208	3160	3160
q14	316	281	281	281
q15	521	480	478	478
q16	497	430	422	422
q17	1791	1499	1504	1499
q18	8098	7933	7905	7905
q19	5538	1536	1391	1391
q20	2082	1828	1809	1809
q21	5102	4833	5005	4833
q22	614	582	556	556
Total cold run time: 77433 ms
Total hot run time: 55602 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173777 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 283ca1f817cba2b71811203d35d808144ca12b43, data reload: false

query1	912	366	364	364
query2	6325	2496	2433	2433
query3	6653	205	216	205
query4	28393	17279	17127	17127
query5	3767	494	473	473
query6	271	177	198	177
query7	4605	294	284	284
query8	337	316	306	306
query9	8623	2388	2363	2363
query10	439	308	260	260
query11	10823	10022	10098	10022
query12	132	85	87	85
query13	1635	375	376	375
query14	10271	7624	6826	6826
query15	228	187	183	183
query16	7583	302	305	302
query17	1390	537	540	537
query18	1781	280	272	272
query19	192	148	150	148
query20	89	79	82	79
query21	206	133	118	118
query22	4600	4107	4134	4107
query23	33922	33601	33567	33567
query24	11894	2906	2879	2879
query25	653	405	396	396
query26	1782	153	151	151
query27	2934	274	281	274
query28	7746	2130	2113	2113
query29	1031	654	638	638
query30	248	154	159	154
query31	971	758	752	752
query32	99	56	57	56
query33	771	311	320	311
query34	948	504	514	504
query35	712	562	564	562
query36	1128	966	1001	966
query37	158	80	83	80
query38	2967	2789	2793	2789
query39	836	823	808	808
query40	240	118	122	118
query41	51	53	51	51
query42	119	94	99	94
query43	588	535	549	535
query44	1191	732	735	732
query45	188	159	166	159
query46	1084	734	698	698
query47	1866	1764	1753	1753
query48	374	305	305	305
query49	916	410	420	410
query50	786	393	397	393
query51	6940	6842	6729	6729
query52	105	97	101	97
query53	358	283	297	283
query54	864	454	445	445
query55	73	73	72	72
query56	286	282	265	265
query57	1106	1021	1030	1021
query58	269	247	251	247
query59	3415	3299	3390	3299
query60	305	288	272	272
query61	96	95	95	95
query62	787	633	640	633
query63	330	290	286	286
query64	10302	2186	1691	1691
query65	3145	3128	3121	3121
query66	1026	342	337	337
query67	15784	15136	14994	14994
query68	5424	545	551	545
query69	617	436	368	368
query70	1205	1231	1119	1119
query71	440	284	295	284
query72	7777	5267	5066	5066
query73	767	325	321	321
query74	5846	5508	5467	5467
query75	3571	2652	2669	2652
query76	3639	902	948	902
query77	623	295	291	291
query78	10393	10938	9333	9333
query79	6528	507	516	507
query80	911	471	472	471
query81	572	224	219	219
query82	295	129	130	129
query83	197	166	165	165
query84	280	91	150	91
query85	714	314	310	310
query86	345	289	313	289
query87	3298	3100	3097	3097
query88	3752	2475	2443	2443
query89	478	397	376	376
query90	1985	187	191	187
query91	129	101	105	101
query92	58	49	48	48
query93	2373	515	511	511
query94	1168	206	215	206
query95	397	319	322	319
query96	603	273	276	273
query97	3240	3027	3041	3027
query98	221	202	202	202
query99	1658	1277	1290	1277
Total cold run time: 290153 ms
Total hot run time: 173777 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.48 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 283ca1f817cba2b71811203d35d808144ca12b43, data reload: false

query1	0.05	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.10	0.08
query5	0.49	0.47	0.48
query6	1.13	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.54	0.48	0.49
query10	0.55	0.54	0.55
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.59
query14	0.77	0.79	0.78
query15	0.85	0.79	0.81
query16	0.35	0.38	0.37
query17	1.06	0.96	0.97
query18	0.22	0.22	0.22
query19	1.79	1.76	1.75
query20	0.01	0.00	0.01
query21	15.42	0.77	0.67
query22	3.94	7.71	1.89
query23	18.30	1.40	1.20
query24	2.05	0.22	0.23
query25	0.16	0.09	0.09
query26	0.29	0.20	0.21
query27	0.46	0.22	0.23
query28	13.31	1.03	1.00
query29	12.57	3.33	3.34
query30	0.25	0.06	0.05
query31	2.87	0.39	0.38
query32	3.28	0.46	0.46
query33	2.92	2.91	2.88
query34	16.71	4.37	4.31
query35	4.42	4.38	4.42
query36	0.65	0.48	0.47
query37	0.19	0.15	0.16
query38	0.15	0.16	0.15
query39	0.04	0.03	0.03
query40	0.15	0.13	0.12
query41	0.10	0.04	0.05
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 109.1 s
Total hot run time: 30.48 s

@ByteYue ByteYue mentioned this pull request Jul 13, 2024
Closed
@ByteYue ByteYue force-pushed the add_sql_support_to_alter_vault branch from 712827e to 8801d07 Compare July 14, 2024 08:20
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

github-actions[bot]
github-actions bot reviewed Jul 14, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@ByteYue
Copy link
Contributor Author

ByteYue commented Jul 14, 2024

run buildall

github-actions[bot]
github-actions bot reviewed Jul 14, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@ByteYue ByteYue force-pushed the add_sql_support_to_alter_vault branch from 8b0f1d0 to 9ea5790 Compare July 15, 2024 08:12
@ByteYue
Copy link
Contributor Author

ByteYue commented Jul 15, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40568 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9ea579066a4c7f2d08152f59368fe0d93f938e3a, data reload: false

------ Round 1 ----------------------------------
q1	18518	4858	4425	4425
q2	2046	198	195	195
q3	10526	1184	1110	1110
q4	10233	766	960	766
q5	7561	2767	2717	2717
q6	228	140	139	139
q7	977	605	606	605
q8	9213	2129	2105	2105
q9	8945	6595	6620	6595
q10	8844	3837	3795	3795
q11	450	243	252	243
q12	397	221	227	221
q13	18406	2989	3018	2989
q14	285	237	240	237
q15	532	490	494	490
q16	492	399	399	399
q17	1002	682	724	682
q18	8391	7651	7564	7564
q19	7028	1471	1363	1363
q20	673	336	320	320
q21	5002	3348	3318	3318
q22	353	293	290	290
Total cold run time: 120102 ms
Total hot run time: 40568 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4405	4287	4320	4287
q2	375	272	286	272
q3	3267	3031	2935	2935
q4	2016	1761	1653	1653
q5	5596	5510	5641	5510
q6	233	137	135	135
q7	2290	1787	1832	1787
q8	3375	3439	3450	3439
q9	8852	8960	8868	8868
q10	4117	3819	3873	3819
q11	614	534	510	510
q12	805	617	651	617
q13	16233	3179	3251	3179
q14	314	295	284	284
q15	549	501	507	501
q16	504	433	445	433
q17	1885	1513	1515	1513
q18	8435	8295	8010	8010
q19	1790	1632	1463	1463
q20	2195	1886	1884	1884
q21	5231	4947	4905	4905
q22	587	538	521	521
Total cold run time: 73668 ms
Total hot run time: 56525 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 174317 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9ea579066a4c7f2d08152f59368fe0d93f938e3a, data reload: false

query1	928	390	367	367
query2	6460	1835	1777	1777
query3	6661	205	218	205
query4	28222	17319	17250	17250
query5	3618	483	492	483
query6	263	180	176	176
query7	4592	291	295	291
query8	247	202	200	200
query9	8567	2379	2347	2347
query10	449	308	278	278
query11	10672	10085	10098	10085
query12	115	87	83	83
query13	1626	368	368	368
query14	10156	7678	7496	7496
query15	226	167	166	166
query16	7590	312	304	304
query17	1816	551	522	522
query18	1938	276	277	276
query19	189	156	162	156
query20	93	85	85	85
query21	207	130	129	129
query22	4365	4043	3995	3995
query23	34019	33522	33603	33522
query24	11103	2939	2896	2896
query25	655	418	393	393
query26	1224	160	160	160
query27	2435	270	276	270
query28	7216	2027	2027	2027
query29	926	647	642	642
query30	260	162	156	156
query31	951	766	738	738
query32	99	57	57	57
query33	754	322	328	322
query34	931	527	517	517
query35	710	594	598	594
query36	1138	1007	1004	1004
query37	169	85	87	85
query38	3003	2895	2843	2843
query39	916	888	825	825
query40	223	133	127	127
query41	48	44	50	44
query42	119	98	100	98
query43	503	468	465	465
query44	1251	723	734	723
query45	204	163	159	159
query46	1098	748	733	733
query47	1885	1793	1743	1743
query48	367	305	293	293
query49	843	415	420	415
query50	794	400	397	397
query51	6854	6814	6781	6781
query52	107	99	97	97
query53	359	295	295	295
query54	883	457	451	451
query55	75	77	75	75
query56	300	302	284	284
query57	1133	1072	1032	1032
query58	264	266	266	266
query59	2741	2574	2631	2574
query60	307	286	285	285
query61	97	94	97	94
query62	793	643	655	643
query63	333	300	296	296
query64	9479	2228	7560	2228
query65	3159	3162	3136	3136
query66	756	332	334	332
query67	15621	15219	14939	14939
query68	4537	536	534	534
query69	628	476	370	370
query70	1206	1155	1201	1155
query71	394	293	289	289
query72	7235	5218	5888	5218
query73	751	329	322	322
query74	6194	5784	5708	5708
query75	3399	2722	2698	2698
query76	2529	988	937	937
query77	653	306	303	303
query78	12891	12832	9367	9367
query79	5828	534	524	524
query80	1134	493	483	483
query81	575	230	232	230
query82	292	137	138	137
query83	337	170	170	170
query84	272	92	93	92
query85	698	367	297	297
query86	460	324	304	304
query87	3287	3122	3110	3110
query88	4540	2443	2441	2441
query89	477	390	384	384
query90	1854	200	195	195
query91	137	100	108	100
query92	65	52	52	52
query93	1520	505	506	505
query94	1118	223	222	222
query95	410	321	318	318
query96	599	270	272	270
query97	3218	3116	3034	3034
query98	235	204	196	196
query99	1538	1257	1291	1257
Total cold run time: 285116 ms
Total hot run time: 174317 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.37 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9ea579066a4c7f2d08152f59368fe0d93f938e3a, data reload: false

query1	0.04	0.04	0.03
query2	0.09	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.09	0.09
query5	0.50	0.50	0.48
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.55	0.48	0.48
query10	0.55	0.55	0.55
query11	0.14	0.11	0.12
query12	0.16	0.13	0.13
query13	0.59	0.59	0.59
query14	0.75	0.79	0.79
query15	0.85	0.81	0.81
query16	0.35	0.37	0.35
query17	1.02	0.99	0.97
query18	0.23	0.22	0.22
query19	1.80	1.74	1.71
query20	0.02	0.01	0.01
query21	15.43	0.74	0.66
query22	4.15	7.62	1.65
query23	18.22	1.34	1.19
query24	2.12	0.22	0.22
query25	0.14	0.09	0.08
query26	0.29	0.21	0.21
query27	0.45	0.23	0.23
query28	13.24	1.02	1.00
query29	12.58	3.38	3.35
query30	0.26	0.06	0.06
query31	2.86	0.39	0.38
query32	3.30	0.48	0.47
query33	2.87	2.91	2.97
query34	17.23	4.40	4.34
query35	4.40	4.39	4.42
query36	0.65	0.47	0.48
query37	0.19	0.16	0.16
query38	0.16	0.16	0.15
query39	0.04	0.03	0.04
query40	0.15	0.13	0.12
query41	0.09	0.05	0.04
query42	0.06	0.04	0.04
query43	0.04	0.05	0.04
Total cold run time: 109.65 s
Total hot run time: 30.37 s

@ByteYue
Copy link
Contributor Author

ByteYue commented Jul 16, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40645 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 93589af849c0247032ea49f26cc31abad95cdc8b, data reload: false

------ Round 1 ----------------------------------
q1	18607	4855	4438	4438
q2	2799	195	203	195
q3	11266	1167	1169	1167
q4	11942	915	815	815
q5	7769	2760	2713	2713
q6	223	139	136	136
q7	967	597	613	597
q8	9334	2127	2118	2118
q9	8946	6655	6582	6582
q10	8844	3767	3821	3767
q11	464	235	242	235
q12	401	222	225	222
q13	17933	3105	3125	3105
q14	295	252	251	251
q15	519	484	494	484
q16	523	391	391	391
q17	1051	723	734	723
q18	8313	7452	7536	7452
q19	6850	1491	1373	1373
q20	689	320	352	320
q21	4897	3269	3881	3269
q22	359	298	292	292
Total cold run time: 122991 ms
Total hot run time: 40645 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4397	4261	4268	4261
q2	392	280	269	269
q3	3030	2808	2750	2750
q4	1880	1653	1662	1653
q5	5360	5333	5362	5333
q6	220	131	130	130
q7	2136	1732	1701	1701
q8	3184	3383	3350	3350
q9	8524	8420	8449	8420
q10	3903	3669	3685	3669
q11	590	497	500	497
q12	767	621	610	610
q13	17448	2988	2990	2988
q14	300	270	272	270
q15	516	481	487	481
q16	483	412	446	412
q17	1805	1499	1480	1480
q18	7765	7578	7440	7440
q19	1685	1453	1626	1453
q20	2030	1788	1769	1769
q21	4898	4764	4831	4764
q22	584	483	484	483
Total cold run time: 71897 ms
Total hot run time: 54183 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173492 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 93589af849c0247032ea49f26cc31abad95cdc8b, data reload: false

query1	913	381	366	366
query2	6468	1906	1865	1865
query3	6670	204	210	204
query4	28373	17438	17438	17438
query5	4189	478	477	477
query6	267	170	169	169
query7	4585	287	282	282
query8	232	194	194	194
query9	8444	2354	2337	2337
query10	433	294	274	274
query11	12544	10041	10025	10025
query12	146	88	83	83
query13	1637	385	370	370
query14	10169	8115	7882	7882
query15	228	166	177	166
query16	7889	328	315	315
query17	1790	553	550	550
query18	1981	291	285	285
query19	205	152	174	152
query20	93	83	81	81
query21	204	137	124	124
query22	4274	4290	3973	3973
query23	33761	33257	33317	33257
query24	11231	2971	2866	2866
query25	624	381	392	381
query26	1112	162	150	150
query27	2679	273	273	273
query28	7014	2002	2003	2002
query29	870	650	652	650
query30	289	151	156	151
query31	975	743	800	743
query32	106	59	55	55
query33	857	294	301	294
query34	921	477	486	477
query35	706	577	555	555
query36	1105	966	968	966
query37	164	79	75	75
query38	2831	2756	2759	2756
query39	859	810	827	810
query40	210	120	117	117
query41	45	46	45	45
query42	114	99	99	99
query43	497	470	455	455
query44	1240	730	726	726
query45	203	161	160	160
query46	1093	780	709	709
query47	1856	1814	1778	1778
query48	378	301	303	301
query49	1081	424	414	414
query50	771	393	385	385
query51	6998	7089	6803	6803
query52	109	96	87	87
query53	355	278	285	278
query54	980	461	450	450
query55	79	72	72	72
query56	321	277	284	277
query57	1113	1040	1059	1040
query58	263	244	282	244
query59	2827	2679	2688	2679
query60	314	286	296	286
query61	98	98	114	98
query62	818	651	655	651
query63	322	288	283	283
query64	9921	2243	1658	1658
query65	3206	3111	3175	3111
query66	987	328	329	328
query67	15372	14981	14880	14880
query68	4698	538	526	526
query69	518	339	320	320
query70	1156	1129	1133	1129
query71	410	284	287	284
query72	8891	5540	5696	5540
query73	762	320	319	319
query74	6239	5649	5715	5649
query75	3962	2684	2680	2680
query76	3169	938	971	938
query77	662	310	314	310
query78	9587	9090	9646	9090
query79	2714	517	524	517
query80	2612	490	488	488
query81	587	222	216	216
query82	903	139	132	132
query83	300	175	164	164
query84	276	95	88	88
query85	2275	310	300	300
query86	498	300	321	300
query87	3282	3099	3121	3099
query88	4750	2454	2482	2454
query89	478	373	427	373
query90	1898	190	195	190
query91	128	101	100	100
query92	64	49	47	47
query93	4473	504	510	504
query94	1262	210	215	210
query95	401	316	314	314
query96	610	275	273	273
query97	3217	3002	3035	3002
query98	226	200	197	197
query99	1724	1259	1252	1252
Total cold run time: 292141 ms
Total hot run time: 173492 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.67 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 93589af849c0247032ea49f26cc31abad95cdc8b, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.68	0.06	0.06
query5	0.51	0.50	0.47
query6	1.13	0.73	0.72
query7	0.02	0.01	0.02
query8	0.06	0.05	0.05
query9	0.55	0.48	0.50
query10	0.56	0.53	0.54
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.60	0.59	0.59
query14	0.76	0.78	0.78
query15	0.85	0.83	0.83
query16	0.36	0.37	0.37
query17	1.05	1.02	1.01
query18	0.23	0.21	0.21
query19	1.86	1.76	1.75
query20	0.02	0.01	0.01
query21	15.40	0.73	0.65
query22	4.01	7.22	1.97
query23	18.33	1.42	1.23
query24	2.08	0.22	0.22
query25	0.16	0.08	0.08
query26	0.30	0.20	0.20
query27	0.45	0.23	0.22
query28	13.30	1.01	0.99
query29	12.63	3.30	3.35
query30	0.25	0.06	0.05
query31	2.87	0.38	0.39
query32	3.28	0.49	0.47
query33	2.88	2.92	2.87
query34	17.03	4.38	4.35
query35	4.46	4.45	4.42
query36	0.64	0.47	0.50
query37	0.18	0.15	0.16
query38	0.15	0.15	0.14
query39	0.04	0.03	0.03
query40	0.16	0.13	0.13
query41	0.09	0.05	0.04
query42	0.06	0.05	0.05
query43	0.04	0.04	0.04
Total cold run time: 109.68 s
Total hot run time: 30.67 s

platoneko
platoneko previously approved these changes Jul 17, 2024
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

gavinchou
gavinchou requested changes Jul 17, 2024
View reviewed changes
Copy link
Contributor

@gavinchou gavinchou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls add regression test for the new feature

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 17, 2024
@ByteYue ByteYue force-pushed the add_sql_support_to_alter_vault branch 5 times, most recently from 3e99939 to 298784b Compare July 22, 2024 09:15
@ByteYue
Copy link
Contributor Author

ByteYue commented Jul 22, 2024

run feut

@ByteYue
Copy link
Contributor Author

ByteYue commented Jul 22, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40184 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 298784b588abdb99d23a71039d417be608eb89ab, data reload: false

------ Round 1 ----------------------------------
q1	18716	4595	4256	4256
q2	2030	196	210	196
q3	10512	1212	1090	1090
q4	10242	782	764	764
q5	7545	2725	2698	2698
q6	224	139	139	139
q7	957	605	608	605
q8	9225	2122	2097	2097
q9	8874	6625	6606	6606
q10	8836	3779	3792	3779
q11	442	244	244	244
q12	395	235	233	233
q13	17753	3025	3018	3018
q14	288	235	228	228
q15	535	480	475	475
q16	487	384	381	381
q17	993	630	658	630
q18	8152	7512	7431	7431
q19	6563	1433	1495	1433
q20	708	330	342	330
q21	5064	3272	3300	3272
q22	346	279	290	279
Total cold run time: 118887 ms
Total hot run time: 40184 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4305	4259	4259
q2	366	277	263	263
q3	3071	2881	2921	2881
q4	1976	1742	1751	1742
q5	5555	5529	5498	5498
q6	224	143	138	138
q7	2296	1835	1829	1829
q8	3574	3463	3431	3431
q9	8821	8860	8863	8860
q10	4164	3688	3827	3688
q11	587	510	499	499
q12	829	631	640	631
q13	16426	3171	3181	3171
q14	307	296	300	296
q15	544	475	503	475
q16	497	443	435	435
q17	1847	1520	1486	1486
q18	8093	8087	7723	7723
q19	1811	1418	1604	1418
q20	2973	1903	1856	1856
q21	8506	4949	4689	4689
q22	615	516	516	516
Total cold run time: 77474 ms
Total hot run time: 55784 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173053 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 298784b588abdb99d23a71039d417be608eb89ab, data reload: false

query1	916	383	374	374
query2	6467	1868	1798	1798
query3	6647	206	216	206
query4	28262	17400	17476	17400
query5	3656	486	490	486
query6	277	173	165	165
query7	4604	286	281	281
query8	235	198	192	192
query9	8539	2468	2431	2431
query10	422	286	269	269
query11	12901	10006	10102	10006
query12	117	86	82	82
query13	1635	370	353	353
query14	8929	6984	7760	6984
query15	213	164	168	164
query16	7077	506	442	442
query17	914	556	549	549
query18	1686	284	273	273
query19	191	151	147	147
query20	88	83	81	81
query21	206	126	130	126
query22	4560	4289	4103	4103
query23	34199	33757	33445	33445
query24	11154	2886	2941	2886
query25	617	418	394	394
query26	700	147	146	146
query27	2332	276	276	276
query28	6121	2083	2075	2075
query29	916	628	644	628
query30	255	154	150	150
query31	979	762	773	762
query32	93	55	54	54
query33	715	343	346	343
query34	908	495	499	495
query35	892	763	753	753
query36	1112	978	955	955
query37	139	84	93	84
query38	3000	2896	2816	2816
query39	913	850	857	850
query40	204	121	118	118
query41	45	43	45	43
query42	117	100	99	99
query43	494	462	465	462
query44	1167	722	716	716
query45	191	158	158	158
query46	1084	741	763	741
query47	1885	1789	1789	1789
query48	360	290	289	289
query49	850	409	420	409
query50	790	391	389	389
query51	6737	6695	6536	6536
query52	101	94	97	94
query53	350	284	282	282
query54	885	485	459	459
query55	77	76	74	74
query56	293	273	269	269
query57	1154	1041	1085	1041
query58	235	256	255	255
query59	2769	2622	2443	2443
query60	306	283	296	283
query61	101	96	95	95
query62	802	639	644	639
query63	316	282	290	282
query64	9536	2274	1754	1754
query65	3173	3085	3156	3085
query66	791	317	339	317
query67	15465	15028	15238	15028
query68	4503	517	530	517
query69	570	435	353	353
query70	1137	1104	1183	1104
query71	382	281	283	281
query72	6873	5545	5480	5480
query73	758	324	325	324
query74	6131	5753	5622	5622
query75	3405	2691	2718	2691
query76	2236	928	935	928
query77	656	299	296	296
query78	9506	9057	8954	8954
query79	3223	503	507	503
query80	2445	465	474	465
query81	597	223	224	223
query82	505	136	137	136
query83	292	177	169	169
query84	274	93	90	90
query85	895	378	308	308
query86	351	312	322	312
query87	3299	3129	3121	3121
query88	3952	2451	2438	2438
query89	492	374	374	374
query90	1859	191	190	190
query91	131	98	103	98
query92	63	49	47	47
query93	4611	505	491	491
query94	1243	303	286	286
query95	403	323	316	316
query96	598	269	272	269
query97	3204	3042	3011	3011
query98	211	196	192	192
query99	1740	1291	1296	1291
Total cold run time: 280410 ms
Total hot run time: 173053 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 298784b588abdb99d23a71039d417be608eb89ab, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.69	0.08	0.07
query5	0.49	0.49	0.49
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.05
query9	0.54	0.48	0.50
query10	0.53	0.53	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.58
query14	0.75	0.79	0.77
query15	0.85	0.83	0.82
query16	0.36	0.37	0.37
query17	0.98	1.03	1.03
query18	0.23	0.22	0.22
query19	1.80	1.80	1.83
query20	0.02	0.01	0.01
query21	15.39	0.78	0.65
query22	4.87	6.27	2.15
query23	18.26	1.48	1.24
query24	2.37	0.24	0.22
query25	0.15	0.09	0.08
query26	0.29	0.21	0.20
query27	0.46	0.23	0.24
query28	13.25	1.02	1.00
query29	12.63	3.32	3.30
query30	0.26	0.06	0.06
query31	2.87	0.39	0.38
query32	3.27	0.47	0.47
query33	2.93	2.93	2.92
query34	17.01	4.38	4.39
query35	4.41	4.42	4.45
query36	0.66	0.46	0.49
query37	0.19	0.16	0.16
query38	0.16	0.16	0.15
query39	0.05	0.04	0.04
query40	0.16	0.12	0.12
query41	0.10	0.05	0.04
query42	0.05	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.53 s
Total hot run time: 31.03 s

@ByteYue
add fe
9ddb534 
parser

add g4

add command

add ms

add builtin op back

extract
@ByteYue ByteYue force-pushed the add_sql_support_to_alter_vault branch from 298784b to 9ddb534 Compare July 22, 2024 15:08
@ByteYue
Copy link
Contributor Author

ByteYue commented Jul 22, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40315 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9ddb53491002c7f61b2d1da1c2bf0d927d677cd3, data reload: false

------ Round 1 ----------------------------------
q1	18859	4592	4333	4333
q2	2016	191	185	185
q3	10530	1186	1173	1173
q4	10204	808	905	808
q5	7586	2721	2700	2700
q6	224	142	136	136
q7	958	596	602	596
q8	9211	2122	2093	2093
q9	8831	6593	6617	6593
q10	8706	3797	3809	3797
q11	453	238	250	238
q12	404	220	220	220
q13	18632	2963	2955	2955
q14	269	232	234	232
q15	521	475	498	475
q16	519	380	386	380
q17	996	695	707	695
q18	8061	7683	7458	7458
q19	5958	1512	1365	1365
q20	656	315	317	315
q21	4884	3277	3306	3277
q22	339	291	291	291
Total cold run time: 118817 ms
Total hot run time: 40315 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4411	4322	4302	4302
q2	382	272	274	272
q3	3114	2902	2888	2888
q4	2003	1787	1755	1755
q5	5533	5556	5482	5482
q6	234	137	131	131
q7	2308	1896	1802	1802
q8	3317	3473	3421	3421
q9	8815	8889	8823	8823
q10	4119	3793	3876	3793
q11	606	503	491	491
q12	832	678	649	649
q13	16386	3152	3183	3152
q14	325	297	286	286
q15	552	502	484	484
q16	498	447	444	444
q17	1862	1528	1508	1508
q18	8174	8115	7875	7875
q19	1839	1613	1494	1494
q20	3034	1880	1886	1880
q21	5176	4967	4679	4679
q22	769	518	519	518
Total cold run time: 74289 ms
Total hot run time: 56129 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173686 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9ddb53491002c7f61b2d1da1c2bf0d927d677cd3, data reload: false

query1	909	372	373	372
query2	6460	1916	1880	1880
query3	6632	208	216	208
query4	24836	17300	17369	17300
query5	3625	470	478	470
query6	274	195	164	164
query7	4590	291	282	282
query8	253	198	194	194
query9	8612	2418	2392	2392
query10	444	301	292	292
query11	10622	10029	10131	10029
query12	121	89	85	85
query13	1642	357	347	347
query14	10194	7678	7740	7678
query15	243	163	166	163
query16	7249	487	473	473
query17	1161	551	517	517
query18	1951	270	272	270
query19	209	148	157	148
query20	88	83	82	82
query21	203	128	124	124
query22	4261	4182	3923	3923
query23	34088	33715	33620	33620
query24	10992	2802	2892	2802
query25	630	420	408	408
query26	720	163	161	161
query27	2263	274	280	274
query28	6076	2064	2062	2062
query29	929	689	627	627
query30	258	154	159	154
query31	1050	783	785	783
query32	92	54	63	54
query33	643	342	339	339
query34	898	480	502	480
query35	868	783	727	727
query36	1110	954	994	954
query37	142	83	84	83
query38	2969	2853	2851	2851
query39	912	866	840	840
query40	209	124	126	124
query41	52	48	47	47
query42	118	108	108	108
query43	530	474	476	474
query44	1102	726	721	721
query45	200	158	162	158
query46	1092	759	739	739
query47	1868	1788	1773	1773
query48	356	292	294	292
query49	841	417	410	410
query50	767	391	397	391
query51	6837	6601	6654	6601
query52	100	97	94	94
query53	358	298	286	286
query54	875	447	448	447
query55	73	73	76	73
query56	288	273	287	273
query57	1118	1072	1068	1068
query58	256	256	243	243
query59	2869	2643	2571	2571
query60	299	283	290	283
query61	101	94	94	94
query62	823	646	650	646
query63	332	305	294	294
query64	9180	2279	1676	1676
query65	3182	3075	3105	3075
query66	781	320	331	320
query67	15443	15021	14830	14830
query68	4526	524	528	524
query69	622	438	378	378
query70	1200	1172	1127	1127
query71	386	290	291	290
query72	7165	5864	5598	5598
query73	756	320	322	320
query74	6147	5691	5722	5691
query75	3411	2765	2676	2676
query76	2207	883	962	883
query77	663	307	322	307
query78	9605	9234	8910	8910
query79	2426	512	511	511
query80	2024	474	484	474
query81	602	220	218	218
query82	886	137	135	135
query83	299	172	164	164
query84	260	83	86	83
query85	1353	307	297	297
query86	480	336	327	327
query87	3248	3113	3144	3113
query88	4162	2338	2348	2338
query89	486	382	391	382
query90	1758	200	195	195
query91	130	106	99	99
query92	62	50	54	50
query93	2214	502	490	490
query94	1097	294	289	289
query95	423	341	340	340
query96	605	278	273	273
query97	3204	3037	3003	3003
query98	222	206	209	206
query99	1794	1272	1290	1272
Total cold run time: 273692 ms
Total hot run time: 173686 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9ddb53491002c7f61b2d1da1c2bf0d927d677cd3, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.04	0.06
query4	1.66	0.08	0.07
query5	0.50	0.48	0.47
query6	1.13	0.74	0.73
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.55	0.49	0.48
query10	0.56	0.54	0.54
query11	0.15	0.12	0.11
query12	0.16	0.11	0.11
query13	0.60	0.58	0.58
query14	0.75	0.80	0.76
query15	0.86	0.82	0.83
query16	0.37	0.37	0.35
query17	1.01	1.00	1.00
query18	0.23	0.22	0.21
query19	1.83	1.70	1.67
query20	0.01	0.00	0.01
query21	15.43	0.82	0.66
query22	4.65	6.68	2.02
query23	18.22	1.36	1.23
query24	2.11	0.24	0.22
query25	0.15	0.09	0.09
query26	0.30	0.21	0.21
query27	0.47	0.23	0.23
query28	13.29	1.01	1.02
query29	12.63	3.37	3.29
query30	0.25	0.06	0.06
query31	2.85	0.39	0.39
query32	3.27	0.48	0.47
query33	2.90	2.91	2.95
query34	16.94	4.35	4.38
query35	4.40	4.38	4.49
query36	0.65	0.47	0.48
query37	0.20	0.16	0.15
query38	0.15	0.14	0.15
query39	0.04	0.03	0.03
query40	0.16	0.12	0.13
query41	0.10	0.05	0.05
query42	0.06	0.05	0.06
query43	0.04	0.04	0.04
Total cold run time: 110.05 s
Total hot run time: 30.63 s

@gavinchou gavinchou merged commit 64b55f0 into apache:master Jul 29, 2024
28 of 31 checks passed
gavinchou pushed a commit that referenced this pull request Sep 10, 2024
@ByteYue @gavinchou
[feature](vault) Support alter storage vault stmt in FE (#37606)
057c252 
Support alter storage vault stmt in FE. Currently only support alter s3
vault's ak and sk.
For example
```sql
alter storage vault s3_vault properties(
"type" = "s3",
"ak" = "new_ak"
);
```
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@gavinchou gavinchou gavinchou approved these changes

@Yukang-Lian Yukang-Lian Yukang-Lian approved these changes

@platoneko platoneko platoneko left review comments

Assignees
No one assigned
Labels
dev/3.0.2-merged meta-change reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ByteYue @doris-robot @gavinchou @platoneko @Yukang-Lian @dataroaring