Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

[hot-fix] increase data-node memory limits to 4Gi #1689

Merged
merged 3 commits into from
Nov 15, 2018
+5 −3
Merged

[hot-fix] increase data-node memory limits to 4Gi #1689

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7d74990
increate data-node memory limits to 4Gi
hao1939 Nov 9, 2018
04cc5d7
increase data-node max java heapsize to 2G
hao1939 Nov 13, 2018
8db01fa
reserve more memory on PAI worker
hao1939 Nov 13, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/hadoop-data-node/deploy/hadoop-data-node.yaml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,12 @@ spec:
periodSeconds: 3
resources:
limits:
memory: "4Gi"
requests:
memory: "1Gi"
env:
- name: HADOOP_DATANODE_OPTS
Copy link
Member

@mzmssg mzmssg Nov 12, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to raise JVM memory opts? I think MT limitation is at JVM level.

Copy link
Contributor Author

@hao1939 hao1939 Nov 12, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, since now we haven't found any java limit was hit.

Copy link
Member

@mzmssg mzmssg Nov 12, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#1658 For 175node screenshot, jave OOM is recorded. So we might need a larger JVM opts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mzmssg ,
What about increase the max heap size to 2G?

value: "-Xmx512m"
value: "-Xmx2048m"
- name: HDFS_ADDRESS
value: {{ clusterinfo[ 'hadoopinfo' ][ 'hadoop_vip' ] }}
- name: GENERATE_CONFIG
Expand Down
4 changes: 2 additions & 2 deletions ...doop-node-manager/deploy/hadoop-node-manager-configuration/nodemanager-generate-script.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ mem_total=`cat /proc/meminfo | grep "MemTotal" | awk '{print $2}'`
# memory size to nodemanager is (mem_total - mem_reserved)
if [ $(grep 'ip:' /host-configuration/host-configuration.yaml|wc -l) -gt 1 ]
then
echo "Node role is 'Worker'. Reserve 12G for os and k8s."
let mem_reserved=12*1024
echo "Node role is 'Worker'. Reserve 16G for os and k8s."
let mem_reserved=16*1024
else
echo "Node role is 'Master & Worker'. Reserve 40G for os and k8s."
let mem_reserved=40*1024
Expand Down