Fix -w

[petdance]

Redo the -w flag to properly handle words.

Update documentation that this affects.

[n1vux]

xref discussion under beyondgrep/ack2#445 Metacharacters and Non-Word chars ; https://github.com/petdance/ack2/issues/565 Unicode (ugh but should have tests to track what DOES work or new enough perls)

[petdance]

This needs #1 for proper testing.

[petdance]

Right now this has been changed to this:

    if ( $opt->{w} ) {
        $str = "\\b(?:$str)\\b";
    }  

Not sure if this is what we want to live with.

[n1vux]

As discussed on prior thread, that handles the commonsense case (where $str is in fact a sensible literal word, or a RE that will match word-chars only) but has nasty edge/corner cases when $str may match whitespace or non-word-punc in first or last position.

root issue is that the \b zerowidth assertion treats non-word printables asymmetrically from word-printable, in that (1)there is no \b within sequente space punc space punc, and (2)the beginning and end of string are Non-word for \b purposes so \b(-foo-)\b will not match in first or last position ever.

we must either document that only the commonsense case where -w means word means word and Warn/Error if pattern $str starts or ends with non-word punc [including metachas; if you want -w $RE you should drop the -w and say which word-effect you want] or define exactly what -w means for $str to match a non-word word in documentation and tests and then implement it.

We discussed some ways using \k etc to make it work in edge cases but depends what we want the edges to be.

[petdance]

I'll be getting at those edge cases today. That's why I didn't close this.

[petdance]

(?:^|\b|\s)\K(?:PATTERN)(?=\s|\b|$) is what we talked about latest. We'll have to see what it does to performance.

[petdance]

We may also want to make an optimization such that if $pattern =~ /^\w+$/ that we use the simple \b$pattern\b.

[patch]

\b{wb} would be better than \b, at least for acking natural language, but it is relatively new to Perl and the semantics are less well-known than programmers’ longstanding expectations of “word boundaries.”

[petdance]

When did it come around? I'm not familiar with it.

[patch]

Perl 5.22 with notable improvements in 5.24.

References:

• perl5220delta: qr/\b{wb}/
• perl5240delta: qr/\b{wb}/ is now tailored to Perl expectations
• perlrebackslash: \b{wb}

[petdance]

Interesting. Thanks for the pointers.

I would never put something in ack where the behavior changes depending on what version of Perl is running.

[patch]

The regex syntax and semantics change in every major Perl release, including those of \b when new Unicode “word” characters are added annually, and therefore those of ack change as well, but I understand your desire for a relatively stable flag.