Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

[hotfix] remove memory limits for k8s services and increase it for rm, zookeeper #1619

Merged
merged 6 commits into from
Oct 30, 2018

Conversation

hao1939
Copy link
Contributor

@hao1939 hao1939 commented Oct 30, 2018

Remove memory limits for k8s services.

Increase memory limits of rm to 36G, max heap size 32G.
Increase memory limits of zookeeper to 10G, max heap size 8G.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 51.638% when pulling 7dd57ba on hao/hot_fix_etcd_memory_limit into 3d0bd96 on pai-0.8.y.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage remained the same at 51.638% when pulling 7dd57ba on hao/hot_fix_etcd_memory_limit into 3d0bd96 on pai-0.8.y.

@coveralls
Copy link

coveralls commented Oct 30, 2018

Coverage Status

Coverage remained the same at 51.638% when pulling 60a73fa on hao/hot_fix_etcd_memory_limit into 3d0bd96 on pai-0.8.y.

@hao1939
Copy link
Contributor Author

hao1939 commented Oct 30, 2018

Hi @fanyangCS ,

I choose 500 nodes on two assumes:

  • 500 nodes should be the biggest cluster user could deploy by themselves, without help from PAI team.
  • most of the PAI clusters are smaller than that.

We could support huge clusters, and a bigger cluster need more advanced configuration. We are not supposed to include that complexity for everyone. Should we?

@hao1939 hao1939 changed the title [hotfix] etcd memory limit [hotfix] remove memory limits for k8s services and increase it for zookeeper Oct 30, 2018
@hao1939 hao1939 requested a review from yqwang-ms October 30, 2018 08:27
@hao1939 hao1939 changed the title [hotfix] remove memory limits for k8s services and increase it for zookeeper [hotfix] remove memory limits for k8s services and increase it for rm, zookeeper Oct 30, 2018
@hao1939 hao1939 merged commit 8a5c224 into pai-0.8.y Oct 30, 2018
hao1939 added a commit that referenced this pull request Nov 15, 2018
…, zookeeper (#1619)

* increate etcd memory limits to 8Gi

* set the "requests" instead of "limits" for k8s services

* increase memory limits for rm

* increase memory limit for zookeeper

* add "requests" for rm

* fix typo
@hao1939 hao1939 mentioned this pull request Nov 15, 2018
hao1939 added a commit that referenced this pull request Nov 16, 2018
* [hotfix] remove memory limits for k8s services and increase it for rm, zookeeper (#1619)

* increate etcd memory limits to 8Gi

* set the "requests" instead of "limits" for k8s services

* increase memory limits for rm

* increase memory limit for zookeeper

* add "requests" for rm

* fix typo

* [hot-fix] increase data-node memory limits to 4Gi (#1689)

* increate data-node memory limits to 4Gi

* increase data-node max java heapsize to 2G

* reserve more memory on PAI worker
@hao1939 hao1939 deleted the hao/hot_fix_etcd_memory_limit branch December 6, 2018 09:19
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants