Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Integrate Dshuttle into PAI #4584

Merged
merged 68 commits into from
Oct 15, 2020
Merged
Changes from 1 commit
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
fd63e4d
update pai version (#4174)
Binyang2014 Feb 10, 2020
5b2caae
[runtime] Only disable job ssh when gangAllocation is set (#4208)
Binyang2014 Feb 17, 2020
491c549
[Runtime] Fix lint and UT (#4214)
Binyang2014 Feb 18, 2020
0ffa794
dshuttle
Binyang2014 Dec 6, 2019
5f1ddb2
deploy change
Binyang2014 Dec 9, 2019
2fdea3d
dshuttle log
Binyang2014 Dec 12, 2019
f51166e
rest server change
Binyang2014 Dec 17, 2019
84e7de2
another update
Binyang2014 Dec 23, 2019
9bdd41f
change to office image
Binyang2014 Dec 27, 2019
b36c7ed
change deploy script
Binyang2014 Apr 10, 2020
5f56a76
add node
Binyang2014 Apr 10, 2020
0849ace
change deploy
Binyang2014 Apr 11, 2020
3964cd6
more change
Binyang2014 Apr 11, 2020
579153c
more changes
Binyang2014 Apr 12, 2020
cab6688
dshuttle udpate
Binyang2014 Apr 13, 2020
e2a6981
dshuttle fix
Binyang2014 Apr 14, 2020
0f5388f
more change
Binyang2014 Apr 22, 2020
5aaa1d3
temp change
Binyang2014 May 6, 2020
3852e1e
config update
Binyang2014 May 9, 2020
95421fc
add fuse sidecar
Binyang2014 May 12, 2020
7b9f55a
add pv/pvc support
Binyang2014 May 26, 2020
6c5e6cb
clean deploy script
Binyang2014 May 27, 2020
ce99e25
rebase master
Binyang2014 May 27, 2020
a76e1e8
orgnize deploy code
Binyang2014 May 27, 2020
da2113c
deploy change
Binyang2014 May 28, 2020
5d962c4
update
Binyang2014 May 28, 2020
36b2656
update image
Binyang2014 May 29, 2020
8ca663d
udpate
Binyang2014 May 29, 2020
c742a55
add delete script
Binyang2014 May 29, 2020
631b485
update
Binyang2014 Jun 3, 2020
f03fcb4
add doc
Binyang2014 Jun 5, 2020
7ec2766
UI change
Binyang2014 Jun 5, 2020
5c2f3ba
fix bug
Binyang2014 Jun 15, 2020
626e441
fix updatedb make disk pressure issue
Binyang2014 Jun 18, 2020
76bb6ef
more changes
Binyang2014 Jul 7, 2020
449f461
more change
Binyang2014 Jul 18, 2020
fb09e4b
merge master
Binyang2014 Jul 30, 2020
8f04d5b
revert change
Binyang2014 Jul 30, 2020
1c070c6
change config
Binyang2014 Jul 30, 2020
21904a9
Merge branch 'master' into binyli/dshuttle
Binyang2014 Aug 5, 2020
c984b71
update config
Binyang2014 Aug 5, 2020
f82ef1c
update config
Binyang2014 Aug 5, 2020
f45fa81
add log4j
Binyang2014 Aug 5, 2020
4d882da
chnage updatedb.conf
Binyang2014 Aug 5, 2020
a2cc20c
add change updatedb command
Binyang2014 Aug 5, 2020
1e5e989
change pylon
Binyang2014 Aug 5, 2020
a8104b5
change pylon config
Binyang2014 Aug 6, 2020
ddfc63b
add dshuttle storage type
Binyang2014 Aug 6, 2020
2d647ff
add doc for deshuttle deploy
Binyang2014 Aug 6, 2020
c39db29
Merge branch 'master' into binyli/dshuttle
Binyang2014 Aug 25, 2020
b9a5453
Merge branch 'master' into binyli/dshuttle
Binyang2014 Sep 8, 2020
0d2b50c
Merge branch 'master' into binyli/dshuttle
Binyang2014 Sep 14, 2020
d70c58a
fix
Binyang2014 Sep 18, 2020
1bf0a90
fix pylon
Binyang2014 Sep 21, 2020
e6a137e
move to acr
Binyang2014 Sep 23, 2020
a0b0b78
fix
Binyang2014 Sep 23, 2020
cd25cab
revert some change
Binyang2014 Sep 23, 2020
5bd73e0
Merge branch 'binyli/dshuttle' of github.com:Microsoft/pai into binyl…
Binyang2014 Sep 23, 2020
9e0ba65
change docker tag
Binyang2014 Sep 23, 2020
fbb1829
fix
Binyang2014 Sep 23, 2020
56d4ab6
fix
Binyang2014 Sep 23, 2020
7711041
update
Binyang2014 Sep 23, 2020
3b62126
fix
Binyang2014 Sep 23, 2020
7c7d547
enhance doc
Binyang2014 Sep 23, 2020
a5d9b89
Add more docs
Binyang2014 Oct 13, 2020
d5b0cec
fix review comments
Binyang2014 Oct 14, 2020
e7c9723
change image version
Binyang2014 Oct 15, 2020
faf9a4e
fix
Binyang2014 Oct 15, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update
Binyang2014 committed Sep 23, 2020
commit 77110411fc0eb8d1653c096f546c7eaaf0e59a93
Original file line number Diff line number Diff line change
@@ -23,4 +23,4 @@ echo "Clean the dshuttle-master's data on the disk"

rm -rf /mnt/journal/* 2>/dev/null
rm -rf /mnt/ufs/* 2>/dev/null
rm -rf /mnt/log/*master*.log 2>/dev/null
rm -rf /mnt/log/*master*.log* 2>/dev/null
Original file line number Diff line number Diff line change
@@ -22,6 +22,6 @@ echo "Clean the dshuttle-wroker's data on the disk"


rm -rf /mnt/ssd/* 2>/dev/null
rm -rf /mnt/log/*worker.log 2>/dev/null
rm -rf /mnt/log/*worker.log* 2>/dev/null


10 changes: 5 additions & 5 deletions src/dshuttle/config/dshuttle.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Dshuttle section parser

- [Default Configuration](#Default-configuration-)
- [How to Configure](#How-to-configure-dshuttle-section-in-service-configurationyaml-)
- [Generated Configuration](#Generated-configuration-)
- [Data Table](#Table-)
- [Notice](#Notice-)
- [Dshuttle section parser](#dshuttle-section-parser)
- [Default configuration](#default-configuration)
- [How to configure Dshuttle section in service-configuration.yaml](#how-to-configure-dshuttle-section-in-service-configurationyaml)
- [Generated Configuration](#generated-configuration)
- [Table](#table)

## Default configuration

Original file line number Diff line number Diff line change
@@ -481,7 +481,7 @@
"selected": false,
"text": "zookeeper",
"value": "zookeeper"
},
}
],
"query": "label_values(service_cpu_percent, name)",
"refresh": 0,
1 change: 0 additions & 1 deletion src/pylon/deploy/pylon-config/location.conf.template
Original file line number Diff line number Diff line change
@@ -93,7 +93,6 @@ location ~ ^/log-manager/([^/]+):(\d+)/(.*)$ {
proxy_pass http://$1:$2/log-manager/$3$is_args$args;
}


{% if CLUSTER_TYPE == 'yarn' %}
#
# YARN web portal