@SET directive

[gagoar]

Altho the documentation talks about some strictness around @set directive, in practice, it is pretty flexible.

• it allows adding the directive anywhere in the file.
• it allows any type of character after the @SET VAR=
• it allows the directive to be in lower case @set as well as @SET

I added some tests to avoid these issues, but these belong to the linter, not the parser.

I've also added a new getter that provides the tokens that are responsible for directives.

[codecov]

Codecov Report

Merging #23 (c4b40eb) into main (98b4503) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main       #23   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            7         7           
  Lines          176       202   +26     
  Branches        30        35    +5     
=========================================
+ Hits           176       202   +26     

Flag	Coverage Δ
unittests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/constants.ts	100.00% <100.00%> (ø)
src/parser.ts	100.00% <100.00%> (ø)
src/schemaToString.ts	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 98b4503...c4b40eb. Read the comment docs.

[patrick-stephens]

A test case not covered is setting the variable multiple times, it appears this is resolved globally before anything else though:

@set A=1
[INPUT]
  name dummy
  dummy {"message":"${A} - 1"}


@SET A=2
[INPUT]
  name dummy
  dummy {"message":"${A} - 2"}

[OUTPUT]
  name stdout
  match *

Gives the output:

[0] dummy.0: [1644408204.453941068, {"message"=>"2 - 1"}]
[1] dummy.0: [1644408205.453826159, {"message"=>"2 - 1"}]
[2] dummy.0: [1644408206.453820301, {"message"=>"2 - 1"}]
[3] dummy.0: [1644408207.453840448, {"message"=>"2 - 1"}]
[0] dummy.1: [1644408204.454052903, {"message"=>"2 - 2"}]
[1] dummy.1: [1644408205.453858461, {"message"=>"2 - 2"}]
[2] dummy.1: [1644408206.453867638, {"message"=>"2 - 2"}]
[3] dummy.1: [1644408207.453871793, {"message"=>"2 - 2"}]

This likely should be a linting rule as well though, it parses fine but it unexpected from a user perspective I think or at least should be flagged the first set is ignore.

[patrick-stephens]

Can we indicate the (first) error here is the space before the equals sign? It seems to be just at the start of the line.

The rules are a bit weird, these are all ok:

@set A= B
@set A=B a
@set A=Baaa=
@set A=Baaa =

It seems to grab everything after the equals sign to the end of the line although it trims any whitespace at the end.

Failures/unexpected

However, these are not:

@set A =B 
@set A = B

This is allowed but actually doesn't set it, it sets a variable called A , i.e. with the space!

@SET A =B
@SET C = D                             

[INPUT]
  name dummy
  dummy {"A":"${A}", "C":"${C}"}

[OUTPUT]
  name stdout
  match *

Gives you the output:
[0] dummy.0: [1644407542.453926584, {"A"=>"", "C"=>""}]

If you tweak it so the space is used in the variable name it gives the expected output:

@SET A =B
@SET C = D                             

[INPUT]
  name dummy
  dummy {"A":"${A }", "C":"${C }"}

[OUTPUT]
  name stdout
  match *

Now we have it: [0] dummy.0: [1644408034.453943425, {"A"=>"B", "C"=>" D"}]

[patrick-stephens]

I think we should have those bottom two checks on the linter though to highlight the fact that the spaces affect things - or a bug in Fluent Bit but it is there for now.

[gagoar]

The last one has to be a bug. I can't even think in how this is not already fixed

[patrick-stephens]

Yeah to me it seems incorrect, however we're currently functioning that way so as a dev tool we should highlight this?

[patrick-stephens]

MIxed case handling for SET and INCLUDE I think is still required.

[patrick-stephens]

Yeah to me it seems incorrect, however we're currently functioning that way so as a dev tool we should highlight this?

[gagoar]

MIxed case handling for SET and INCLUDE I think is still required.

Can you give me an example? what is not doing now?

[patrick-stephens]

MIxed case handling for SET and INCLUDE I think is still required.

Can you give me an example? what is not doing now?

So, just need it to correctly parse the following:

@Set A=A
@SeT B=B
@SEt C=C
@seT D=D
@sET E=E
@sEt F=F

Apologies if it does this already but looking at the code it only seems to uppercase set then match on SET.
Similarly for include, just need a few examples of mixed case to confirm.

[gagoar]

>
now I understand what you meant. I will add this test and make the adjustments!

[gagoar]

@patrick-stephens I've implemented the case-sensitive directives. It took me a minute given that the moo tokenizer doesn't let you make regex insensitive 😢 .

There's a refactoring pending to make the tokenizer a bit more ergonomic (code-wise) but I prefer to do a fast-follow when this is landed.