wip allow for op suffixes

[KristofferC]

Ref JuliaLang/julia#22089 (comment)

[KristofferC]

@ZacLN As discussed with @Keno, there are two ways to handle this.

One is either as in this PR, to have a custom "operator suffix" kind (I called it badly SPEC_OP here but we can fix name later) that we emit as a separate token after the standard operator.
The other is to effectively double the number of kinds such that each operator has an associated special operator. I think I like the way done in this PR more because it reduces the number of kinds and the code churn here, but perhaps there is a difference in how well it translates to the parsing code.

[ZacLN]

My intuition is that this is the better approach

[codecov-io]

Codecov Report

Merging #117 into master will decrease coverage by 0.65%.
The diff coverage is 30.76%.

@@            Coverage Diff             @@
##           master     #117      +/-   ##
==========================================
- Coverage   76.15%   75.49%   -0.66%     
==========================================
  Files           4        4              
  Lines         847      857      +10     
==========================================
+ Hits          645      647       +2     
- Misses        202      210       +8

Impacted Files	Coverage Δ
src/token.jl	73.77% <100%> (ø)	⬆️
src/lexer.jl	85.54% <25%> (-1.02%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b3c8238...952f882. Read the comment docs.

[Keno]

@ZacLN @KristofferC Are we good with this approach? If so, shall we merge?

[ZacLN]

Can I have a think on how to implement it in cstparser first? I'll have a look this weekend

[Keno]

Bump!

[ZacLN]

Sorry for delay.

Thoughts- this doesn't do the job properly. There's no ambiguity in these cases and no reliance on context (above what is known at the Tokenize level) so we should be returning a single Token.

Similarly, do the changes in v0.7 make use of dot operators unambiguous? i.e. should they be handled to return a single TOken at this level?

EDIT: Further thoughts: introduce an extra function call before emitting an operator that checks for suffixes and appends it to the val field + possibly add a Boolean field to Token/RawToken indicating its not a 'pure' operator?

[ZacLN]

An example for dotted operator handling handling
#128

[Keno]

Superseded by #129