Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]Pod kb-addon-prometheus-server in CrashLoopBackOff because too many open files #2244

Closed
ahlfors opened this issue Mar 27, 2023 · 1 comment
Assignees
Labels
bug kind/bug Something isn't working
Milestone

Comments

@ahlfors
Copy link

ahlfors commented Mar 27, 2023

Describe the bug
Pod kb-addon-prometheus-server in CrashLoopBackOff because too many open files

To Reproduce
Steps to reproduce the behavior:

  1. Install kbcli and init playground;
  2. Create mysql cluster;
  3. Keep the cluster running for a week;
  4. Check pod kb-addon-prometheus-server status
  5. See error:
    Pod kb-addon-prometheus-server in CrashLoopBackOff
    Defaulted container "prometheus-server-configmap-reload" out of: prometheus-server-configmap-reload, prometheus-server
    2023/03/27 11:27:28 too many open files

Expected behavior
Pod kb-addon-prometheus-server in Running status

Desktop (please complete the following information):

OS: [Linux 5.10.0-21-cloud-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux]

Additional context
prometheus-server CrashLoopBackOff
#15742

@ahlfors ahlfors added the kind/bug Something isn't working label Mar 27, 2023
@ahlfors ahlfors assigned ahlfors and unassigned nayutah Mar 27, 2023
@nashtsai nashtsai added this to the Release 0.5.0 milestone Apr 19, 2023
@nashtsai nashtsai assigned yimeisun and unassigned ahlfors May 10, 2023
@yimeisun yimeisun assigned ahlfors and unassigned yimeisun May 10, 2023
@ahlfors
Copy link
Author

ahlfors commented May 10, 2023

This issue exist on gcp vm, because its default process file size limits is too small 1024, while aws ec2 default process file size limits is 65535, ok for this case. User need to adjust the vm process file size limits before running kubeblocks. Close issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants