Parser improvements: != operator and key: [line_break]

[aibaars]

This pull request implements two fixes to the parser:

• parsing of x!=1 (used to be parsed as x!(1) instead of x != 1)
• handling of line breaks after a hash-key symbol, for example

foo x:
   "some text"

should be parsed as foo(x: "some text") but it was parsed as foo(x:); "some text" because the parser incorrectly interpreted the line break after x: as a statement terminator.

The first problem is addressed by letting the scanner handle identifiers that end with !, so it can do a negative lookahead test for a = character.

The second problem is addressed by adding a fake "no_line_break" token in the grammar right after the : symbol in a pair and change the scanner so it won't produce "line_break" symbols if a "no_line_break" symbol is expected. This solution feels a bit "hacky" to me, but it could not think of a better way. @maxbrunsfeld any ideas?

Checklist:

• All tests pass in CI.
• There are sufficient tests for the new fix/feature.
• Grammar rules have not been renamed unless absolutely necessary.
• The conflicts section hasn't grown too much.
• The parser size hasn't grown too much (check the value of STATE_COUNT in src/parser.c).

[aryx]

was this semicolon intended here?

[aibaars]

Yes it is intended. Without the semicolon the test case was not valid Ruby code. The tree-sitter grammar used to accept that code without semicolon, but it really shouldn't have.