[Perl] Refactor whole syntax definition to drop Oniguruma

[deathaxe]

Fixes #541, #1029, #1325, #1326, #1495

Supersedes #1556

According to #481 this PR provides an Oniguruma free Perl syntax definition written from scratch.

OUTLINE

1. fixes all currently open issues
2. much more feature complete
3. completely compatible with new regex engine
4. uses embed to include custom syntaxes within POD and HEREDOCS (CSS,JS,JSON,HTML,SQL,XML)
5. provides a syntax test file
6. uses as much atomic rules from Perl.sublime-syntax and
   ModernPerl package as possible (with some fixes/modifications)
7. follows the doc http://perldoc.perl.org as close as possible
8. add some snippets with some files renamed to match naming style

REMARKS

1. The benchmark tests on a 100k file with all kinds of statements output about 360ms for evaluation on a Windows box.

2. The main focus is to create a robust base for syntax highlighting rather than scoping each language entity/construct in detail.

   a) Not all meta scoping RFCs are implemented in detail due to some limitations caused by HEREDOC. Especially pushing in/out of meta.block or meta.braces breaks some HEREDOC constructs.

   b) Scope names were changed to follow the scope naming guidelines as close as possible and try to use as general names as possible.
   -> Maybe needs some refines?

3. At the current point the implementation is a work in progress and needs testing by perl power users to

   a) check for highlighting to work in all situations

   b) constants/variables/functions are scoped correctly

4. This commit adds some snippets but does not include any completions in favor of https://packagecontrol.io/packages/Perl%20Completions

[glenntanner3]

Forgive my noobness: How can I use this? Do I have to compile all of sublime or are there some config files I can update? Thank you, I could really use the perl fixes. s/n// and s/n/m/gr are driving me CRAZY.

[deathaxe]

What I do to look into single packages is:

1. git clone this repo to some folder outside ST. (e.g.: C:\Data\ST\Packages)
2. GitExtensions for Windows has a function to list and download pull requests, which are checked out as local branches.
3. Create a symlink (linux/osx) or junction (windows) of C:\Data\ST\Packages\Perl in %USERPROFILE%\Appdata\Sublime Text 3\Packages.

The location of your ST's Packages depends on OS and setup.

You could also clone https://github.com/deathaxe/Packages and checkout the pr/perl branch.

This way I can modify the package and test syntaxes while keeping in sync with the repo.

Maybe easiest way

If you just want to checkout the Perl.sublime-syntax, all you need to do is to copy the RAW content of current snapshot and put the content into Packages/Perl/Perl.sublime-syntax

[michaelblyons]

This might be overkill for @glenntanner3, but I maintain an "octopus" branch that contains all the fixes (local or remote) that I want to use before they are merged. Then I set up a git alias that says

[alias]
    octo = "!git fetch --all && \
             git checkout octopus && \
             git reset --hard upstream/master && \
             git merge -m 'git-octo merge' \
                       thom/javascript-template-keymap \
                       thom/javascript-cleanup-regexps \
                       fictefoll/pr/python-string-modifier-auto-pairing \
                       stealthy/diff_newline_at_eof \
                       deathaxe/pr/perl \
                       c#/string-interpolation \
                       c#/index-properties"

Then, if something interesting happens to upstream/master or one of the branches, I just run git octo, and I have all the updates to all of the branches I care about.¹ If I want to add someone's new PR, I put it in the list. If a PR gets merged, I take it out.

¹ Merge conflicts between branches are not handled gracefully. I usually pick the one I'd rather have.

[glenntanner3]

Thanks for the recommendations. I'm brand new to sublime so still learning how it works. I will give these a try upon return from vacation. PS: Linux

[wbond]

This sounds great. I did not go through it in detail, but I know @deathaxe has quite a bit of experience with the default syntaxes, and we can gather user feedback through the next dev cycle.

[keith-hall]

I have a small suggestion to ensure reliability/compatibility behavior is maintained

[keith-hall]

presumably this space should be enclosed in a char class, else it could break as soon as this variable is used in expressions with extended mode enabled

[deathaxe]

Which is actually the case in the pragma context without negative impact.

[deathaxe]

... but there is another reason, why a change might be useful. The current solution doesn't respect tabs.