More fix cursor for duplicate header #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gfoltz
reviewed
Aug 21, 2023
|
Also, why not put this branch/PR on the papaparse repo itself? Is this just for early feedback? |
gfoltz
approved these changes
Aug 21, 2023
|
Yes, wanted internal discussion before submitting to the main repo |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Papaparse's parse method modifies in the input string when headers are modified and/or duplicated. This changes the overall length of the input.
This change in length needs to be accounted for in the adjustment of lastCursor in pushRow. This is so that the outer parseChunk function can properly account for how much of the chunk is leftover and needs to be kept for prefixing onto the next chunk.
There was a previous attempt to fix this issue, but it only worked for the fast parse path. This fix works for both fast parsing and not-fast.
The added test correctly fails without the fix.
I've also included a bug fix for inputLen, which is needed to correctly handle a file that end with a quote and has modified headers or duplicate headers. I'll add a test to prove this works too.