Generate headers in Parser, based upon presence of _fields, without changing _input (fixes issue #985)

[landisdesign]

Fixes issue #985.

Previously, headers were created in ParseHandle's processResults(), but also managed by Parser.parse() when baseIndex was falsy.

When headers were requested and baseIndex was false, parse() would modify _input without adjusting cursor, causing the parser to mistake where it needed to continue parsing. This would cause further rows to be shifted. In addition, baseIndex could be undefined or 0, depending on how it was called, causing this cursor shift to potentially happen twice.

This PR:

• Updates _fields based upon the value of _results.meta.fields without changing _input
• Uses _fields.length as the source of truth for whether or not headers should be generated when config.header is truthy
• Consolidates header generation within Parser by passing _fields to the Parser constructor where the above test is performed
• Uses the same parsing algorithms used for row generation for headers
• Adds fields to results generated by Parser.parse() based upon whether or not headers are available

Additional impacts:

• Because header generation is now performed within pushRow(), the header row is no longer returned as data by Parser.parse(). Instead, it is returned in meta.fields by returnable(). This also means we don't need to adjust config.preview to account for the header row.

[landisdesign]

Changes to existing tests were required to reflect that:

• Parser now requires an array as its second argument for its constructor
• Parser.parse() no longer returns the header row as data

[beckyconning]

🎉

[beckyconning]

Hey @landisdesign

Can't thank you enough.

PapaParse basically didn't work from in a stream pipeline without this.

You are a life saver.

[beckyconning]

@mholt is there a different plan to resolve this issue? it seems quite pressing and this pr seems to resolve it nicely.

[mholt]

As long as it works, and the tests pass, this is OK with me -- will probably defer to @pokoli for a closer look though. Ping me again if it isn't merged in a while.

[pokoli]

The changes also look good to me, I added some comments to simplify the the tests a little bit. Could you have a look at it?

[pokoli]

Could you please add new tests at the end?

[landisdesign]

Yup! Done.

[pokoli]

Do we really need to have 5 rows? Can we make the test simpler with less rows?

[landisdesign]

Probably not. I reduced it to two rows and four columns, enough to make sure we could test column value duplicates and that the headers didn't shift between pause/resume stages.

[pokoli]

Why pusshing headers multiple times? Won't be better to just push when empty?

[landisdesign]

I wanted to double-check that multiple passes returned the same headers. It looked like the original code could have updated the headers on each pass, which seemed problematic. Perhaps that question makes sense as a separate test, or do you think it isn't worth the check?