Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enh(elixir) Improves and fixes many issues with Elixir grammar #3212

Merged
merged 11 commits into from
Jun 5, 2021
Merged

enh(elixir) Improves and fixes many issues with Elixir grammar #3212

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
76a58ed
Use correct multiline string syntax in test file
angelikatyborska May 30, 2021
1385181
Support private macros
angelikatyborska May 30, 2021
4d17178
Color `defstruct` as a class definition
angelikatyborska May 30, 2021
740119b
Merge remote-tracking branch 'origin/main' into elixir-maintenance
angelikatyborska Jun 5, 2021
5f78052
Add aliases with Elixir file extensions
angelikatyborska Jun 5, 2021
67c5e97
Remove incorrect Elixir keywords
angelikatyborska Jun 5, 2021
e929414
Add missing Elixir keywords
angelikatyborska Jun 5, 2021
969b834
Extract literals from keywords
angelikatyborska Jun 5, 2021
c90b2f4
Make defprotocol a keyword
angelikatyborska Jun 5, 2021
729dec8
Fix detecting numbers with leading zeros
angelikatyborska Jun 5, 2021
3605ef9
Fill out CHANGES.md
angelikatyborska Jun 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 22 additions & 22 deletions src/languages/elixir.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,46 +13,45 @@ export default function(hljs) {
const ELIXIR_IDENT_RE = '[a-zA-Z_][a-zA-Z0-9_.]*(!|\\?)?';
const ELIXIR_METHOD_RE = '[a-zA-Z_]\\w*[!?=]?|[-+~]@|<<|>>|=~|===?|<=>|[<>]=?|\\*\\*|[-/+%^&*~`|]|\\[\\]=?';
const KEYWORDS = [
"alias",
"after",
"alias",
"and",
"begin",
"break",
"case",
"catch",
"cond",
"defined",
"defstruct",
"do",
"else",
"end",
"ensure",
"false",
"fn",
"for",
"if",
"import",
"in",
"include",
"module",
"next",
"nil",
"not",
"or",
"quote",
"redo",
"raise",
"receive",
"require",
"retry",
"return",
"self",
"then",
"true",
"reraise",
"rescue",
"try",
"unless",
"until",
"unquote",
"unquote_splicing",
"use",
"when",
"while",
"with|0"
];
const LITERALS = [
"false",
"nil",
"true"
];
const KWS = {
$pattern: ELIXIR_IDENT_RE,
keyword: KEYWORDS
keyword: KEYWORDS,
literal: LITERALS
};
const SUBST = {
className: 'subst',
Expand All @@ -62,7 +61,7 @@ export default function(hljs) {
};
const NUMBER = {
className: 'number',
begin: '(\\b0o[0-7_]+)|(\\b0b[01_]+)|(\\b0x[0-9a-fA-F_]+)|(-?\\b[1-9][0-9_]*(\\.[0-9_]+([eE][-+]?[0-9]+)?)?)',
begin: '(\\b0o[0-7_]+)|(\\b0b[01_]+)|(\\b0x[0-9a-fA-F_]+)|(-?\\b[0-9][0-9_]*(\\.[0-9_]+([eE][-+]?[0-9]+)?)?)',
relevance: 0
};
// TODO: could be tightened
Expand Down Expand Up @@ -218,7 +217,7 @@ export default function(hljs) {
};
const FUNCTION = {
className: 'function',
beginKeywords: 'def defp defmacro',
beginKeywords: 'def defp defmacro defmacrop',
end: /\B\b/, // the mode is ended by the title
contains: [
hljs.inherit(hljs.TITLE_MODE, {
Expand Down Expand Up @@ -272,6 +271,7 @@ export default function(hljs) {

return {
name: 'Elixir',
aliases: ['ex', 'exs'],
joshgoebel marked this conversation as resolved.
Show resolved Hide resolved
keywords: KWS,
contains: ELIXIR_DEFAULT_CONTAINS
};
Expand Down
40 changes: 40 additions & 0 deletions test/markup/elixir/conditionals.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
<span class="hljs-keyword">case</span> x <span class="hljs-keyword">do</span>
<span class="hljs-number">1</span> -&gt; <span class="hljs-symbol">:one</span>
<span class="hljs-number">2</span> -&gt; <span class="hljs-symbol">:two</span>
_ -&gt; <span class="hljs-symbol">:error</span>
<span class="hljs-keyword">end</span>

<span class="hljs-keyword">cond</span> <span class="hljs-keyword">do</span>
x &gt; <span class="hljs-number">30</span> -&gt; <span class="hljs-symbol">:ok</span>
y &lt;= <span class="hljs-number">7</span> -&gt; <span class="hljs-symbol">:maybe</span>
z == <span class="hljs-symbol">:skip</span> -&gt; <span class="hljs-symbol">:ok</span>
<span class="hljs-literal">true</span> -&gt; <span class="hljs-symbol">:error</span>
<span class="hljs-keyword">end</span>

<span class="hljs-keyword">if</span> x &gt; <span class="hljs-number">4</span> <span class="hljs-keyword">do</span>
<span class="hljs-symbol">:ok</span>
<span class="hljs-keyword">end</span>

<span class="hljs-keyword">if</span> x &gt; <span class="hljs-number">4</span>, <span class="hljs-symbol">do:</span> <span class="hljs-symbol">:ok</span>

<span class="hljs-keyword">if</span> x &gt; <span class="hljs-number">4</span> <span class="hljs-keyword">do</span>
<span class="hljs-symbol">:ok</span>
<span class="hljs-keyword">else</span>
<span class="hljs-symbol">:error</span>
<span class="hljs-keyword">end</span>

<span class="hljs-keyword">if</span> x &gt; <span class="hljs-number">4</span>, <span class="hljs-symbol">do:</span> <span class="hljs-symbol">:ok</span>, <span class="hljs-symbol">else:</span> <span class="hljs-symbol">:error</span>

<span class="hljs-keyword">unless</span> y &lt; <span class="hljs-number">50</span> <span class="hljs-keyword">do</span>
<span class="hljs-symbol">:error</span>
<span class="hljs-keyword">end</span>

<span class="hljs-keyword">unless</span> y &lt; <span class="hljs-number">50</span>, <span class="hljs-symbol">do:</span> <span class="hljs-symbol">:error</span>

<span class="hljs-keyword">unless</span> y &lt; <span class="hljs-number">50</span> <span class="hljs-keyword">do</span>
<span class="hljs-symbol">:error</span>
<span class="hljs-keyword">else</span>
<span class="hljs-symbol">:ok</span>
<span class="hljs-keyword">end</span>

<span class="hljs-keyword">unless</span> y &lt; <span class="hljs-number">50</span>, <span class="hljs-symbol">do:</span> <span class="hljs-symbol">:error</span>, <span class="hljs-symbol">else:</span> <span class="hljs-symbol">:ok</span>
40 changes: 40 additions & 0 deletions test/markup/elixir/conditionals.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
case x do
1 -> :one
2 -> :two
_ -> :error
end

cond do
x > 30 -> :ok
y <= 7 -> :maybe
z == :skip -> :ok
true -> :error
end

if x > 4 do
:ok
end

if x > 4, do: :ok

if x > 4 do
:ok
else
:error
end

if x > 4, do: :ok, else: :error

unless y < 50 do
:error
end

unless y < 50, do: :error

unless y < 50 do
:error
else
:ok
end

unless y < 50, do: :error, else: :ok
6 changes: 6 additions & 0 deletions test/markup/elixir/function-title.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,10 @@

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">f!</span></span>, <span class="hljs-symbol">do:</span> IO.puts <span class="hljs-string">&quot;hello world&quot;</span>

<span class="hljs-function"><span class="hljs-keyword">defp</span> <span class="hljs-title">f?</span></span>, <span class="hljs-symbol">do:</span> <span class="hljs-literal">true</span>

<span class="hljs-function"><span class="hljs-keyword">defmacro</span> <span class="hljs-title">foo</span></span>, <span class="hljs-symbol">do:</span> <span class="hljs-symbol">:ok</span>

<span class="hljs-function"><span class="hljs-keyword">defmacrop</span> <span class="hljs-title">do_foo</span></span>, <span class="hljs-symbol">do:</span> <span class="hljs-symbol">:ok</span>

x = <span class="hljs-number">5</span>
6 changes: 6 additions & 0 deletions test/markup/elixir/function-title.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,10 @@ end

def f!, do: IO.puts "hello world"

defp f?, do: true

defmacro foo, do: :ok

defmacrop do_foo, do: :ok

x = 5
12 changes: 12 additions & 0 deletions test/markup/elixir/modules.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<span class="hljs-class"><span class="hljs-keyword">defmodule</span> <span class="hljs-title">User</span></span> <span class="hljs-keyword">do</span>
<span class="hljs-keyword">defstruct</span> [<span class="hljs-symbol">:name</span>, <span class="hljs-symbol">:email</span>, <span class="hljs-symbol">age:</span> <span class="hljs-number">18</span>]
<span class="hljs-keyword">end</span>

<span class="hljs-class"><span class="hljs-keyword">defprotocol</span> <span class="hljs-title">Size</span></span> <span class="hljs-keyword">do</span>
<span class="hljs-variable">@doc</span> <span class="hljs-string">&quot;Calculates the size (and not the length!) of a data structure&quot;</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">size</span></span>(data)
<span class="hljs-keyword">end</span>

<span class="hljs-class"><span class="hljs-keyword">defimpl</span> <span class="hljs-title">Size</span></span>, <span class="hljs-symbol">for:</span> Map <span class="hljs-keyword">do</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">size</span></span>(map), <span class="hljs-symbol">do:</span> map_size(map)
<span class="hljs-keyword">end</span>
12 changes: 12 additions & 0 deletions test/markup/elixir/modules.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
defmodule User do
defstruct [:name, :email, age: 18]
end

defprotocol Size do
@doc "Calculates the size (and not the length!) of a data structure"
def size(data)
end

defimpl Size, for: Map do
def size(map), do: map_size(map)
end
7 changes: 7 additions & 0 deletions test/markup/elixir/numbers.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,21 @@
integer = <span class="hljs-number">1234</span>
integer_with_leading_zero = <span class="hljs-number">01234</span>
integer_zero = <span class="hljs-number">0</span>
big_integer = <span class="hljs-number">1_234_000</span>
neg_integer = <span class="hljs-number">-20_000</span>
float = <span class="hljs-number">2.34</span>
float_with_leading_zero = <span class="hljs-number">0.34</span>
float_zero = <span class="hljs-number">0.0</span>
sci_float = <span class="hljs-number">2.4e23</span>
plus_sci_float = <span class="hljs-number">2.4e+23</span>
small_sci_float = <span class="hljs-number">2.4e-23</span>
cap_sci_float = <span class="hljs-number">2.4E23</span>
binary = <span class="hljs-number">0b1010</span>
binary_with_leading_zero = <span class="hljs-number">0b0010</span>
strange_binary = <span class="hljs-number">0b1010_1010_1010</span>
octal = <span class="hljs-number">0o777</span>
octal_with_leading_zero = <span class="hljs-number">0o077</span>
strange_octal = <span class="hljs-number">0o777_666_555</span>
hex = <span class="hljs-number">0x1ABEF</span>
hex_with_leading_zero = <span class="hljs-number">0x0ABEF</span>
strange_hex = <span class="hljs-number">0x1234_FACE_987D</span>
7 changes: 7 additions & 0 deletions test/markup/elixir/numbers.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,21 @@
integer = 1234
integer_with_leading_zero = 01234
integer_zero = 0
big_integer = 1_234_000
neg_integer = -20_000
float = 2.34
float_with_leading_zero = 0.34
float_zero = 0.0
sci_float = 2.4e23
plus_sci_float = 2.4e+23
small_sci_float = 2.4e-23
cap_sci_float = 2.4E23
binary = 0b1010
binary_with_leading_zero = 0b0010
strange_binary = 0b1010_1010_1010
octal = 0o777
octal_with_leading_zero = 0o077
strange_octal = 0o777_666_555
hex = 0x1ABEF
hex_with_leading_zero = 0x0ABEF
strange_hex = 0x1234_FACE_987D
8 changes: 6 additions & 2 deletions test/markup/elixir/strings.expect.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
a = <span class="hljs-string">&quot;&quot;&quot;test&quot;&quot;&quot;</span>
b = <span class="hljs-string">&#x27;&#x27;&#x27;test&#x27;&#x27;&#x27;</span>
a = <span class="hljs-string">&quot;&quot;&quot;
test
&quot;&quot;&quot;</span>
b = <span class="hljs-string">&#x27;&#x27;&#x27;
test
&#x27;&#x27;&#x27;</span>
c = <span class="hljs-string">&quot;test&quot;</span>
d = <span class="hljs-string">&#x27;test&#x27;</span>
8 changes: 6 additions & 2 deletions test/markup/elixir/strings.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
a = """test"""
b = '''test'''
Copy link
Member

@joshgoebel joshgoebel May 31, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see how these are better examples (that you added), but if the previous were valid we should also leave them to serve as additional regression tests... one can still use a multi-line string for a single line - that is valid grammar, yes?

Copy link
Contributor Author

@angelikatyborska angelikatyborska May 31, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is valid grammar, yes?

No, those were syntax errors in Elixir. The only allowed characters after the opening '''/""" is optional whitespace, and then a newline is required.

My assumption is that it's not a big deal if the syntax highlighting colors something that doesn't compile so I didn't feel the need to change the regexes for multiline strings, but the test examples should be valid Elixir code, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, those were syntax errors in Elixir.

Whoa. I stand corrected. ;-)

My assumption is that it's not a big deal if the syntax highlighting colors something that doesn't compile

Right, it's not our job to detect invalid code.

but the test examples should be valid Elixir code, right?

For the most part, yes. In this particular case this indeed makes sense. I just see people "swap" test cases often when they should be adding new cases instead, [not removing old ones]. Thought this was perhaps yet another case of that. :)

a = """
test
"""
b = '''
test
'''
c = "test"
d = 'test'