Regex: start line character ignored when in group

[MischaDeNola]

Have to move start line marker outside of a group in order for it to get the respect it deservers.

-        rule /(^!)(.*$\n?)/ do #doesn't work
+        rule /^(!)(.*$\n?)/ do #totally works
          groups Generic::Subheading, Generic::Strong
        end

Even without groups, just putting the entire regex in a single group rule /(^!.*$\n?)/, Generic::Strong will cause it to disrespect the ^. I couldn't find anything about Ruby regex that explains this, which is why I'm reporting it as either a bug in ruby, or in my ability to google.

---

Potentially useful longer post

@jneen Hi, I'm a writer, not a developer, so my apologies for sounding like an idiot:

A couple years ago I wrote a custom lexer for rouge for my company's own proprietary language. I recently added a diff rule (for strong) per revathskumar's example so that we could draw attention to particular lines in code blocks when writing instructional docs.

To keep customers from copying the ! from our code blocks, I split the labeling with the groups thing:

        rule /(^!)(.*$\n?)/ do
          groups Generic::Subheading, Generic::Strong
        end

I then just set the style for Subheading to display: none; and it worked like a charm. We highlight the line while excluding the bang. Yay!

However, I recently discovered a bug with this in a couple of rare situations where the rule would trigger inappropriately--not respecting the start-of-line assertion.

function(
domain!whatever(
string: "hello"
)
)

Would render that as:

function(
domainwhatever(
string: "hello"
)
)

When I removed the groups, it did NOT trigger on domain!whatever:

rule /^!.*$\n?/, Generic::Strong

I don't know why using groups makes it act like this. I'm working around the problem by just adding a rule to catch domain!whatever.

        rule /(^!)(.*$\n?)/ do
          groups Generic::Subheading, Generic::Strong
        end
+        rule /(domain!\[a-zA-Z_][a-zA-Z0-9_]*)/, Str # this stops weird bug

Just in case this is an actual bug, I'm letting you know.

Thank you for your time (and for rouge!)

[jneen]

Ah, yes. It is a known problem that traces back to a StringScanner bug that I opend a loooong time ago: https://bugs.ruby-lang.org/issues/7092. Most of the time we work around it by detecting \n and pushing a :bol state that has a default :pop! at the end - see C for an example (preprocessor macros like #pragma etc use this approach)

[jneen]

We sort of work around the above by detecting if a regex's source begins with ^, and if so manually checking #beginning_of_line?, but as you have found it doesn't work if you put it anywhere else in the regex.

See https://github.com/jneen/rouge/blob/master/lib/rouge/regex_lexer.rb#L19 and https://github.com/jneen/rouge/blob/master/lib/rouge/regex_lexer.rb#L297.

[MischaDeNola]

Ok, good to know, thanks! I was losing confidence in my regex mastery; as a writer, it's the only thing I can do better than the engineers here.

[jneen]

@jneen Hi, I'm a writer, not a developer, so my apologies for sounding like an idiot:

Also, just so you know you didn't sound like an idiot at any point :]. This is a weird and subtle bug that I'm sure we'll have to reference many times in the future.

[MischaDeNola]

Thanks! Not to toot my own horn too much, but I have copy-and-pasted others' code for many years now. With enough time and fiddling, something usually happens. As I like to say around here, "Whatever you can do in an hour, I can do in two days."

I'm just waiting for the right dev job at this point--"Looking for someone with extraordinary regex skills, mind-boggling functional spreadsheets, but just a faint glimmer of Groovy, Python, and Ruby."

[stale]

This contribution has been automatically marked as stale because it has not had any activity for more than a year. It will be closed if no additional activity occurs within the next 14 days.