Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Leak related to many DNS lookup failures #59

Open
yaauie opened this issue Dec 11, 2019 · 1 comment
Open

Memory Leak related to many DNS lookup failures #59

yaauie opened this issue Dec 11, 2019 · 1 comment
Labels

Comments

@yaauie
Copy link
Contributor

yaauie commented Dec 11, 2019

We have seen multiple reports of a memory leak, and have so far traced it down to the exceptions being raised by the underlying Resolv library when it cannot perform a lookup or a reverse-lookup.

An analysis of the memory dump finds that the exceptions raised by the resolv library and immediately caught by this plugin are not being dereferenced as we would expect them to, and end up flagged as the cause of a subsequent exception, producing an endless chain from the most recent exception to the first.

I believe the root issue to be beyond the scope of this plugin, as the exceptions are rescue'd and immediately allowed to fall out of scope, but that the volume of exceptions raised by this implementation when it is failing many lookups may exacerbate the issue.

Reports:

  • v3.0.13 (Logstash 7.5.0)
  • v3.0.12 (Logstash 7.1.1)
@yaauie yaauie added the crashes label Dec 11, 2019
@yaauie
Copy link
Contributor Author

yaauie commented Dec 16, 2019

While chasing down the incorrect chaining of exceptions should definitely happen, I believe we have lower hanging fruit for this plugin.

If we change this filter to not use exceptions for lookup misses (as opposed to lookup failures like timeouts or nameserver connection errors), which will drastically reduce the number of exceptions handled and at least slow the rate of the memory leak. I have filed a ticket to track that work: #60

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant