Improve ldiff binary files detection #104
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The former code was performing scan on the first 4K of each file to see if one of them has a '\0' char in it and consider it as a binary file. This commit does not change this heuristic just the implementation. Instead of using the scan method with a regexp, use a simple include?. This not only fix compatibility issues with UTF8 escape sequences, but also the performance: 1. it does not leverage a Regexp system. 2. it stops at first occurence worst case is O(n). 3. it does not store much. Also instead of using .empty? which would signal a non-binary file, the call to include? invert the boolean test. IMHO it is clearer. Note: this could have been achieved simply by replacing .empty by .any? but the other improvements listed above motivated the change.
halostatue
approved these changes
Jan 29, 2025
This was
linked to
issues
Feb 2, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The former code was performing scan on the first 4K of each file to see if one of them has a '\0' char in it and consider it as a binary file.
This commit does not change this heuristic just the implementation. Instead of using the
scanmethod with a regexp, use a simpleinclude?.This not only fix compatibility issues with UTF8 escape sequences (see #102), but also the performance:
Also instead of using
empty?which would signal a non-binary file, the call toinclude?invert the boolean test.IMHO it is clearer.
Note: this could have been achieved simply by replacing
empty?byany?but the other improvements listed above motivated the change.