add test for jinja template test

[GordonWang]

• clean up the logic about sampleDir and jinja_template_dir setting
• Add functional test for jinja template generator
• Fixes [BUG] can not use conditional expression in for loop in the jinja template #174
• Fixes [BUG] default jinja template dir is not correctly resolved when sampleDir is set in conf file #167

[GordonWang]

this part is changed to get the sampleDir according to the conf file directory.

[GordonWang]

we need to trim the newline and empty lines for the output of jinja template.
Because for each validate event, we assume it is a one line json string which has the format

{ 'time': xxx, 'raw': xxx, 'sourcetype': sss} 

skipping empty lines and breaking the loop is the root cause of #174

[lephino]

how do I do multiline events then? We don't want to trim "every" "\n" just the ones where the only thing in the line is "\n".

[lephino]

I see the issue and get what you're trying to solve, but I don't think we can just use .strip(), gotta be another way

[GordonWang]

To generate multiline events in the ‘raw’ field in one json formatted events, use can do this in the jinja template directly. Just like this

{% for _ in range(0, large_number) %}
{%- time_now -%}
 {"_time":"{{ time_now_epoch }}", "_raw":"{{ time_now_formatted }}this is the first line, seq: {{loop.index}}/{{large_number}}\nthis is the second line, seq: {{loop.index}}/{{large_number}}\nthis is the thrid line, seq: {{loop.index}}/{{large_number}}"}
{% endfor %}

the output of this jinja template is

2019-04-24T10:47:35this is the first line, seq: 1/4
this is the second line, seq: 1/4
this is the thrid line, seq: 1/4
2019-04-24T10:47:35this is the first line, seq: 2/4
this is the second line, seq: 2/4
this is the thrid line, seq: 2/4
2019-04-24T10:47:35this is the first line, seq: 3/4
this is the second line, seq: 3/4
this is the thrid line, seq: 3/4
2019-04-24T10:47:35this is the first line, seq: 4/4
this is the second line, seq: 4/4
this is the thrid line, seq: 4/4

Make sense?

[lephino]

I should also say, are we sure we want to force people to write this single line, instead of letting them just hit return on the template? https://github.com/splunk/eventgen/blob/develop/tests/sample_eventgen_conf/jinja/templates/test_jinja_loop.template#L4. Specifically, doesn't it make sense to just hit return and put them on a new line?

[GordonWang]

No worries. User can hit the enter key to make the new line directly in jinja template file.
Jinja engine will keep what it is in your template file and render the output. No changes for newline and whitespace.

So, this template

{% for _ in range(0, large_number) %}
{%- time_now -%}
  {"_time":"{{ time_now_epoch }}", "_raw":"{{ time_now_formatted }}this is the first line, seq: {{loop.index}}/{{large_number}} 
    this is the second line, seq: {{loop.index}}/{{large_number}} 
    this is the thrid line, seq: {{loop.index}}/{{large_number}}"}
{% endfor %}

the output is

2019-04-24T15:58:23this is the first line, seq: 1/4
    this is the second line, seq: 1/4
    this is the thrid line, seq: 1/4
2019-04-24T15:58:23this is the first line, seq: 2/4
    this is the second line, seq: 2/4
    this is the thrid line, seq: 2/4
2019-04-24T15:58:23this is the first line, seq: 3/4
    this is the second line, seq: 3/4
    this is the thrid line, seq: 3/4
2019-04-24T15:58:23this is the first line, seq: 4/4
    this is the second line, seq: 4/4
    this is the thrid line, seq: 4/4

[GordonWang]

describe how we resolve sampleDir

[GordonWang]

describe how to resolve jinja_template_dir

[li-wu]

Verified on my local env.

[GordonWang]

all FT passed.