Fix syntax is ignore after multi-line comment

[GreasySlug]

Fixes #311.

A one-line comment was executed after a multi-line comment.
That's why, syntax after multi-line comments was ignored.

There may also be a whitespace after the multi-line comment.
This could be invalid, please check.

a =
#[comment]#    1#Is this valid?

I made it syntax error.

Changes proposed in this PR:

• Fix syntax is ignore after multi-line comment

@mtshiba

[mtshiba]

I would like to make multiline comment grammatically equivalent to space. This follows other languages. So, a comment at the beginning of a sentence is a syntax error.

i #[: Int]# = 1
↓
i = 1

#[]# print! "a"
↓
 print! "a" # syntax error

[GreasySlug]

I would like to make multiline comment grammatically equivalent to space. This follows other languages. So, a comment at the beginning of a sentence is a syntax error.

Does the length of the whitespace change depending on the content of the multi-line comment?
Or one whitespace regardless of content?

#[]# a = 1
#[sample]# b = 1
↓
[ ]a = 1
[ ]b = 1
or
[ ]a = 1
[       ]b = 1

[mtshiba]

Since comments, like spaces, are not tokens, their length is grammatically meaningless.

Comments have the same meaning as spaces, but they are not replaced by an actual spaces.

However, its length must be taken into account when dealing with token locations.

[GreasySlug]

Yes, there is indeed a problem of misalignment caused by comments.
I'll start a separate Issue for this.