[bugfix] handle extracting non-conformant zip files

[silentrald]

Issue described in discord

• Change extract logic in import maps.
• Remove jszip and node-stream-zip packages.

NOTES:

• Used yauzl since that support non-conformant zip files and ram efficiency.
  • Caveats, may slow down zip extraction a little bit.
• Added test on standard and the breaking map zip file.
• Removed jszip in getting oculus manifest and I can't test it on my end.
• Ready for review but will need to address some comments (BUG and NOTE).
• Do we need to add more tests on big zip files, or a sample exported map files to see the memory usage?

[Zagrios]

@silentrald I think you should go with yauzl 🤔 That way, we can also use it for the map import feature (and the zip files used when importing maps can be very large, so JSZip is a big no no in this case 😅)

[Zagrios]

@silentrald Everything seems to be working fine, except when I import multiple maps from a zip; it never ends:

I haven’t had time to debug, but the issue likely comes from these lines:

if (progress.current === progress.total) {
    completeNewFolder();
}

But, overall, the modifications seems a bit over engineered to me 😅
I have some ideas to simplify zip manipulations; I'll test them tomorrow to see if they're feasible and whether they actually make zip manipulations easier or not 😅 Otherwise we will probably stay like that 👌

[silentrald]

@Zagrios yeah its a little bit over 😅 but was trying to not expose the yauzl module to everything so that its testable, so whenever a change happens, its should be isolated within zip.helpers.

Although I encountered that in coding and I suspect its in the Subject that I was using in importMaps. Sometimes it would drop next events and when that happens, the current != total occurs. I tested on mine and works with 89 songs but ill try to exhaustively add more songs to check if I can replicate.

[silentrald]

Tested on around ~1,400 maps and its working fine 🤔 so its very weird that its not failing on my end

[silentrald]

Notes:

• Never though of generators 😯 but seems to be working, only had to set autoClose: false to make everything work as expected.
• Just a weird bug with path.matchesGlob doesnt support escaping special characters, just used regex for this instead.
• Maybe consider changing the name of YauzlZip to another name class like BsmExtractorZip since in theory, we own/control the function calls on that class. (In the future we can create BsmGeneratorZip if we need to).
• sonarqube might need to ignore test case files.
  • Couldn't get sonar.exclusions=src/__tests__/** to work, might need to be configured in the hosting ig.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Zagrios]

🙌

[Zagrios]

Thanks for the help @silentrald ! ❤️
Everything's working fine 👌
Merging 🚀