Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[oxlog] skip unnecessary lstat #7372

Merged
merged 2 commits into from
Jan 21, 2025

Conversation

papertigers
Copy link
Contributor

@papertigers papertigers commented Jan 17, 2025

We should not incur the cost of calling lstat on every log file oxlog walks over when we are returning all log files regardless of size. If someone requests that we don't show 0 length files through oxlog the binary or the library then we will do the lstat in order to check the file length.

I originally found this edge case when operating on a sled with 600k plus log files that were being walked. Figuring out what we want to do about aging out old debug logs is out of the scope of this PR.

Created using spr 1.3.6-beta.1
Copy link
Contributor

@andrewjstone andrewjstone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks @papertigers!

@papertigers papertigers enabled auto-merge (squash) January 17, 2025 19:52
@papertigers papertigers disabled auto-merge January 17, 2025 20:28
Created using spr 1.3.6-beta.1
@papertigers
Copy link
Contributor Author

The test failure here looks a lot like #7255 but there was a fix for that. Poking some before I rerun the failed test...

@papertigers papertigers enabled auto-merge (squash) January 21, 2025 19:08
@papertigers papertigers merged commit 4285881 into main Jan 21, 2025
16 checks passed
@papertigers papertigers deleted the spr/papertigers/oxlog-skip-unnecessary-lstat branch January 21, 2025 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants