Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] can not use conditional expression in for loop in the jinja template #174

Closed
GordonWang opened this issue Apr 19, 2019 · 1 comment · Fixed by #177
Closed

[BUG] can not use conditional expression in for loop in the jinja template #174

GordonWang opened this issue Apr 19, 2019 · 1 comment · Fixed by #177
Assignees
Labels
bug Malfunctioning Eventgen due to potential bug

Comments

@GordonWang
Copy link
Contributor

Describe the bug
when using if/else expression in for loop in jinja template, the for loop does not work.

To Reproduce
Steps to reproduce the behavior:

  1. Use the eventgen.conf attached and the jinja template attached.
  2. Generate the events with splunk_eventgen command line tool.
  3. Check the output events.

Expected behavior
There should be 9 events.

Actual behavior
There are only 3 events. One event for each generator iteration.

Screenshots
N/A

Sample files and eventgen.conf file

Do you run eventgen with SA-eventgen?
No

If you are using eventgen with pip module mode (please complete the following information):

  • python version: 2.7
  • OS: Mac os 10.14
  • Virtual Env is used: Yes/No
  • Eventgen Version 6.3.4

Additional context

@GordonWang GordonWang added the bug Malfunctioning Eventgen due to potential bug label Apr 19, 2019
@GordonWang
Copy link
Contributor Author

Configuration file content

[test_jinja_template_advance]
end = 1
count = 3
interval = 5
sampleDir = tests/large/sample
jinja_template_dir = tests/large/sample/templates
generator = jinja
jinja_target_template = test_jinja_tmpl_advance.j2
jinja_variables = {"large_number":3, "username": "admim", "flag": true}
earliest = -3s
latest = now
outputMode = stdout

Jinja template file

{% for idx in [1,2,3] %}
{%- time_slice earliest="1234", latest="2345", count=loop.index, slices="5" -%}
{% if flag %}
{"_time":"{{ time_target_epoch }}", "_raw":"{{ time_target_formatted }} [{{username}}] test jinja template generator, switch={{flag}}, seq: {{loop.index}}/{{large_number}}"}
{% else %}
{"_time":"{{ time_target_epoch }}", "_raw":"{{ time_target_formatted }} [{{username}}] test jinja template generator, switch={{flag}}, seq: {{loop.index}}/{{large_number}}"}
{% endif %}
{% endfor %}

GordonWang added a commit that referenced this issue Apr 23, 2019
Add functional test for jinja template generator
Fixes #174
Fixes #167
GordonWang added a commit that referenced this issue Apr 24, 2019
* clean up the logic about sampleDir and jinja_template_dir setting
Add functional test for jinja template generator
Fixes #174
Fixes #167

* revert the change about token, sample mode test cases

* merge the change from develop branch

* merge test conf from develop branch

* use urllib 1.24.2

* format with pep8

* fix bug in template dir resolution

* update test case to address multiline event
li-wu added a commit that referenced this issue May 8, 2019
* Fix jinja template bug under SA-Eventgen app

* Feature timeMultiple (#141)

* Adjusting interval with timeMultiple

* Update issue templates

add the bug report and feature request issue templates

* changing the stanzas to produce data

* Windbag generator/count + end=0 edge cases (#145)

* Generator handling for count = -1, end = 0 working properly

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* [Docs] update contribute docs (#148)

* [Docs] update contribute docs

* [Docs] update the contribute

* Fix make docs bug and summary anchor link error

* init unittest

* [Build] add ut for timeparser functions

* Add more UT for config module

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* Update docs

* Post-PEP8 Fixes (#157)

* skip sort, gitignore 3rd party libs

* fixed yapf ignore, ran format across repo

* Issue 160 (#163)

* Fixed timer and token

* Added a conditional for end=-1

* Update eventgentimer.py

* Fixed timer and token (#162)

* add extendIndexes feature (#154)

* add extendIndexes feature

* set extendIndexes as a list value

* correct log level

* upate doc, change num to weight

* Test fix (#168)

* Add sample functional test for replay mode

* Add token replacement functional tests

* skip failed case

* Added a timeout

* created a results dir

* Update version.json

* Fix previous pep8 format regression bug (#171)

* Fix merge conflict bug

* Delete tool orca related part (#178)

* add test for jinja template test (#177)

* clean up the logic about sampleDir and jinja_template_dir setting
Add functional test for jinja template generator
Fixes #174
Fixes #167

* revert the change about token, sample mode test cases

* merge the change from develop branch

* merge test conf from develop branch

* use urllib 1.24.2

* format with pep8

* fix bug in template dir resolution

* update test case to address multiline event

* fix a issue that when setup eventgen with 200+ indexers, any exceptions will terminate setup and re-writing default configure files

* Fix bug #179 urlparams is not set (#181)

* fix issue-183

* add modinput ft (#185)

* Issue 159 (#186)

* Fixed timer and token

* Added a conditional for end=-1

* Added timeMultiple logic

* Time Multiple Fix

* Add default pull request reviewers (#190)

* Default to -1 (#189)

* Changed verbose -> verbosity (#191)

* Update README.md (#195)

Like other Splunk products - Splunk Enterprise Security, Splunk Business Workflow ... - Splunk Event Generator does not need a definitive article "The" before the product name.

* update doc for friendly reading and add backward capability section. (#193)

* Update index.md (#194)

Update index.md

* change the sampleDir setting in test cases (#196)

* return status=2 when all tasks finished. (#182)

* return status=2 when all tasks finished.

* Change release verion to 6.3.6 (#200)

* Change release verion to 6.3.6
li-wu added a commit that referenced this issue May 10, 2019
* Fix jinja template bug under SA-Eventgen app

* Feature timeMultiple (#141)

* Adjusting interval with timeMultiple

* Update issue templates

add the bug report and feature request issue templates

* changing the stanzas to produce data

* Windbag generator/count + end=0 edge cases (#145)

* Generator handling for count = -1, end = 0 working properly

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* [Docs] update contribute docs (#148)

* [Docs] update contribute docs

* [Docs] update the contribute

* Fix make docs bug and summary anchor link error

* init unittest

* [Build] add ut for timeparser functions

* Add more UT for config module

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* Update docs

* Post-PEP8 Fixes (#157)

* skip sort, gitignore 3rd party libs

* fixed yapf ignore, ran format across repo

* Issue 160 (#163)

* Fixed timer and token

* Added a conditional for end=-1

* Update eventgentimer.py

* Fixed timer and token (#162)

* add extendIndexes feature (#154)

* add extendIndexes feature

* set extendIndexes as a list value

* correct log level

* upate doc, change num to weight

* Test fix (#168)

* Add sample functional test for replay mode

* Add token replacement functional tests

* skip failed case

* Added a timeout

* created a results dir

* Update version.json

* Fix previous pep8 format regression bug (#171)

* Fix merge conflict bug

* Delete tool orca related part (#178)

* add test for jinja template test (#177)

* clean up the logic about sampleDir and jinja_template_dir setting
Add functional test for jinja template generator
Fixes #174
Fixes #167

* revert the change about token, sample mode test cases

* merge the change from develop branch

* merge test conf from develop branch

* use urllib 1.24.2

* format with pep8

* fix bug in template dir resolution

* update test case to address multiline event

* fix a issue that when setup eventgen with 200+ indexers, any exceptions will terminate setup and re-writing default configure files

* Fix bug #179 urlparams is not set (#181)

* fix issue-183

* add modinput ft (#185)

* Issue 159 (#186)

* Fixed timer and token

* Added a conditional for end=-1

* Added timeMultiple logic

* Time Multiple Fix

* Add default pull request reviewers (#190)

* Default to -1 (#189)

* Changed verbose -> verbosity (#191)

* Update README.md (#195)

Like other Splunk products - Splunk Enterprise Security, Splunk Business Workflow ... - Splunk Event Generator does not need a definitive article "The" before the product name.

* update doc for friendly reading and add backward capability section. (#193)

* Update index.md (#194)

Update index.md

* change the sampleDir setting in test cases (#196)

* return status=2 when all tasks finished. (#182)

* return status=2 when all tasks finished.

* Change release verion to 6.3.6 (#200)

* Change release verion to 6.3.6

* Fix flaky tests (#204)

* Fix flaky tests

* Rename modinput to mod_input to avoid functional test fail
li-wu added a commit that referenced this issue Jun 5, 2019
* Fix jinja template bug under SA-Eventgen app

* Feature timeMultiple (#141)

* Adjusting interval with timeMultiple

* Update issue templates

add the bug report and feature request issue templates

* changing the stanzas to produce data

* Windbag generator/count + end=0 edge cases (#145)

* Generator handling for count = -1, end = 0 working properly

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* [Docs] update contribute docs (#148)

* [Docs] update contribute docs

* [Docs] update the contribute

* Fix make docs bug and summary anchor link error

* init unittest

* [Build] add ut for timeparser functions

* Add more UT for config module

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* Update docs

* Post-PEP8 Fixes (#157)

* skip sort, gitignore 3rd party libs

* fixed yapf ignore, ran format across repo

* Issue 160 (#163)

* Fixed timer and token

* Added a conditional for end=-1

* Update eventgentimer.py

* Fixed timer and token (#162)

* add extendIndexes feature (#154)

* add extendIndexes feature

* set extendIndexes as a list value

* correct log level

* upate doc, change num to weight

* Test fix (#168)

* Add sample functional test for replay mode

* Add token replacement functional tests

* skip failed case

* Added a timeout

* created a results dir

* Update version.json

* Fix previous pep8 format regression bug (#171)

* Fix merge conflict bug

* Delete tool orca related part (#178)

* add test for jinja template test (#177)

* clean up the logic about sampleDir and jinja_template_dir setting
Add functional test for jinja template generator
Fixes #174
Fixes #167

* revert the change about token, sample mode test cases

* merge the change from develop branch

* merge test conf from develop branch

* use urllib 1.24.2

* format with pep8

* fix bug in template dir resolution

* update test case to address multiline event

* fix a issue that when setup eventgen with 200+ indexers, any exceptions will terminate setup and re-writing default configure files

* Fix bug #179 urlparams is not set (#181)

* fix issue-183

* add modinput ft (#185)

* Issue 159 (#186)

* Fixed timer and token

* Added a conditional for end=-1

* Added timeMultiple logic

* Time Multiple Fix

* Add default pull request reviewers (#190)

* Default to -1 (#189)

* Changed verbose -> verbosity (#191)

* Update README.md (#195)

Like other Splunk products - Splunk Enterprise Security, Splunk Business Workflow ... - Splunk Event Generator does not need a definitive article "The" before the product name.

* update doc for friendly reading and add backward capability section. (#193)

* Update index.md (#194)

Update index.md

* change the sampleDir setting in test cases (#196)

* return status=2 when all tasks finished. (#182)

* return status=2 when all tasks finished.

* Change release verion to 6.3.6 (#200)

* Change release verion to 6.3.6

* Fix flaky tests (#204)

* Fix flaky tests

* Rename modinput to mod_input to avoid functional test fail

* when hit an exception, make sure egx logs it to files

* Fix circleci status badges (#208)

* Clean up and interval error (#211)

* Fix generatorWorks not work issue (#207)

* Fix navigation error with installed with splunk stream (#214)

* add metric_httpevent plugin

* update log content in metric_httpevent

* add doc for metric_httpevent

* Add 3rd lib in app (#210)

* Add 3rd lib in app

* Bugfix/197 multiprocess not working (#218)

* Fix issue 197 multiprocess not working

* fix issue 219 (#220)

* define httpevent_core and add fix eventgen-httpeventout log handler

* restore samples file

* restore unit test file

* add metric_httpevent plugin

* update log content in metric_httpevent

* add doc for metric_httpevent

* define httpevent_core and add fix eventgen-httpeventout log handler

* restore samples file

* restore unit test file

* Add license credits (#222)

* Feature/multi indexes (#224)

* Fix jinja template bug under SA-Eventgen app

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* add extendIndexes feature

* set extendIndexes as a list value

* upate doc, change num to weight

* calculate generate rate and use sequential index replacement

* update dockerfile

* randomize index replacement in each batch

* Fix jinja template bug under SA-Eventgen app

* Update docs (#146)

* Updated docs, added release notes to changelog

* add python code style lint and format

* Add more unit tests for config module

* calculate generate rate and use sequential index replacement

* update dockerfile

* randomize index replacement in each batch

* fix no len dict out of range

* clean duplicate code

* Revert "Metrics output plugin" (#226)

* [issue 217]disable logging queue in multiprocess mode (#223)

* disable logging queue in multiprocess mode

* Fixed fileName (#229)

* Issue 201 (#221)

* Removing unavailable servers

* Removed nonresponding httpserver

* Added a counter

* fixed a bug

* Added docs and spec

* Addressing unused var

* fixing url reference

* fixed currentreadsize

* Httpevent str formatting

* fix #166 (#192)
li-wu added a commit that referenced this issue Jun 5, 2019
* Fix jinja template bug under SA-Eventgen app

* Feature timeMultiple (#141)

* Adjusting interval with timeMultiple

* Update issue templates

add the bug report and feature request issue templates

* changing the stanzas to produce data

* Windbag generator/count + end=0 edge cases (#145)

* Generator handling for count = -1, end = 0 working properly

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* [Docs] update contribute docs (#148)

* [Docs] update contribute docs

* [Docs] update the contribute

* Fix make docs bug and summary anchor link error

* init unittest

* [Build] add ut for timeparser functions

* Add more UT for config module

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* Update docs

* Post-PEP8 Fixes (#157)

* skip sort, gitignore 3rd party libs

* fixed yapf ignore, ran format across repo

* Issue 160 (#163)

* Fixed timer and token

* Added a conditional for end=-1

* Update eventgentimer.py

* Fixed timer and token (#162)

* add extendIndexes feature (#154)

* add extendIndexes feature

* set extendIndexes as a list value

* correct log level

* upate doc, change num to weight

* Test fix (#168)

* Add sample functional test for replay mode

* Add token replacement functional tests

* skip failed case

* Added a timeout

* created a results dir

* Update version.json

* Fix previous pep8 format regression bug (#171)

* Fix merge conflict bug

* Delete tool orca related part (#178)

* add test for jinja template test (#177)

* clean up the logic about sampleDir and jinja_template_dir setting
Add functional test for jinja template generator
Fixes #174
Fixes #167

* revert the change about token, sample mode test cases

* merge the change from develop branch

* merge test conf from develop branch

* use urllib 1.24.2

* format with pep8

* fix bug in template dir resolution

* update test case to address multiline event

* fix a issue that when setup eventgen with 200+ indexers, any exceptions will terminate setup and re-writing default configure files

* Fix bug #179 urlparams is not set (#181)

* fix issue-183

* add modinput ft (#185)

* Issue 159 (#186)

* Fixed timer and token

* Added a conditional for end=-1

* Added timeMultiple logic

* Time Multiple Fix

* Add default pull request reviewers (#190)

* Default to -1 (#189)

* Changed verbose -> verbosity (#191)

* Update README.md (#195)

Like other Splunk products - Splunk Enterprise Security, Splunk Business Workflow ... - Splunk Event Generator does not need a definitive article "The" before the product name.

* update doc for friendly reading and add backward capability section. (#193)

* Update index.md (#194)

Update index.md

* change the sampleDir setting in test cases (#196)

* return status=2 when all tasks finished. (#182)

* return status=2 when all tasks finished.

* Change release verion to 6.3.6 (#200)

* Change release verion to 6.3.6

* Fix flaky tests (#204)

* Fix flaky tests

* Rename modinput to mod_input to avoid functional test fail

* when hit an exception, make sure egx logs it to files

* Fix circleci status badges (#208)

* Clean up and interval error (#211)

* Fix generatorWorks not work issue (#207)

* Fix navigation error with installed with splunk stream (#214)

* add metric_httpevent plugin

* update log content in metric_httpevent

* add doc for metric_httpevent

* Add 3rd lib in app (#210)

* Add 3rd lib in app

* Bugfix/197 multiprocess not working (#218)

* Fix issue 197 multiprocess not working

* fix issue 219 (#220)

* define httpevent_core and add fix eventgen-httpeventout log handler

* restore samples file

* restore unit test file

* add metric_httpevent plugin

* update log content in metric_httpevent

* add doc for metric_httpevent

* define httpevent_core and add fix eventgen-httpeventout log handler

* restore samples file

* restore unit test file

* Add license credits (#222)

* Feature/multi indexes (#224)

* Fix jinja template bug under SA-Eventgen app

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* add extendIndexes feature

* set extendIndexes as a list value

* upate doc, change num to weight

* calculate generate rate and use sequential index replacement

* update dockerfile

* randomize index replacement in each batch

* Fix jinja template bug under SA-Eventgen app

* Update docs (#146)

* Updated docs, added release notes to changelog

* add python code style lint and format

* Add more unit tests for config module

* calculate generate rate and use sequential index replacement

* update dockerfile

* randomize index replacement in each batch

* fix no len dict out of range

* clean duplicate code

* Revert "Metrics output plugin" (#226)

* [issue 217]disable logging queue in multiprocess mode (#223)

* disable logging queue in multiprocess mode

* Fixed fileName (#229)

* Issue 201 (#221)

* Removing unavailable servers

* Removed nonresponding httpserver

* Added a counter

* fixed a bug

* Added docs and spec

* Addressing unused var

* fixing url reference

* fixed currentreadsize

* Httpevent str formatting

* fix #166 (#192)

* Change release version to 6.4.0 and add release notes (#232)
li-wu added a commit that referenced this issue Jun 5, 2019
* Fix jinja template bug under SA-Eventgen app

* Feature timeMultiple (#141)

* Adjusting interval with timeMultiple

* Update issue templates

add the bug report and feature request issue templates

* changing the stanzas to produce data

* Windbag generator/count + end=0 edge cases (#145)

* Generator handling for count = -1, end = 0 working properly

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* [Docs] update contribute docs (#148)

* [Docs] update contribute docs

* [Docs] update the contribute

* Fix make docs bug and summary anchor link error

* init unittest

* [Build] add ut for timeparser functions

* Add more UT for config module

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* Update docs

* Post-PEP8 Fixes (#157)

* skip sort, gitignore 3rd party libs

* fixed yapf ignore, ran format across repo

* Issue 160 (#163)

* Fixed timer and token

* Added a conditional for end=-1

* Update eventgentimer.py

* Fixed timer and token (#162)

* add extendIndexes feature (#154)

* add extendIndexes feature

* set extendIndexes as a list value

* correct log level

* upate doc, change num to weight

* Test fix (#168)

* Add sample functional test for replay mode

* Add token replacement functional tests

* skip failed case

* Added a timeout

* created a results dir

* Update version.json

* Fix previous pep8 format regression bug (#171)

* Fix merge conflict bug

* Delete tool orca related part (#178)

* add test for jinja template test (#177)

* clean up the logic about sampleDir and jinja_template_dir setting
Add functional test for jinja template generator
Fixes #174
Fixes #167

* revert the change about token, sample mode test cases

* merge the change from develop branch

* merge test conf from develop branch

* use urllib 1.24.2

* format with pep8

* fix bug in template dir resolution

* update test case to address multiline event

* fix a issue that when setup eventgen with 200+ indexers, any exceptions will terminate setup and re-writing default configure files

* Fix bug #179 urlparams is not set (#181)

* fix issue-183

* add modinput ft (#185)

* Issue 159 (#186)

* Fixed timer and token

* Added a conditional for end=-1

* Added timeMultiple logic

* Time Multiple Fix

* Add default pull request reviewers (#190)

* Default to -1 (#189)

* Changed verbose -> verbosity (#191)

* Update README.md (#195)

Like other Splunk products - Splunk Enterprise Security, Splunk Business Workflow ... - Splunk Event Generator does not need a definitive article "The" before the product name.

* update doc for friendly reading and add backward capability section. (#193)

* Update index.md (#194)

Update index.md

* change the sampleDir setting in test cases (#196)

* return status=2 when all tasks finished. (#182)

* return status=2 when all tasks finished.

* Change release verion to 6.3.6 (#200)

* Change release verion to 6.3.6

* Fix flaky tests (#204)

* Fix flaky tests

* Rename modinput to mod_input to avoid functional test fail

* when hit an exception, make sure egx logs it to files

* Fix circleci status badges (#208)

* Clean up and interval error (#211)

* Fix generatorWorks not work issue (#207)

* Fix navigation error with installed with splunk stream (#214)

* add metric_httpevent plugin

* update log content in metric_httpevent

* add doc for metric_httpevent

* Add 3rd lib in app (#210)

* Add 3rd lib in app

* Bugfix/197 multiprocess not working (#218)

* Fix issue 197 multiprocess not working

* fix issue 219 (#220)

* define httpevent_core and add fix eventgen-httpeventout log handler

* restore samples file

* restore unit test file

* add metric_httpevent plugin

* update log content in metric_httpevent

* add doc for metric_httpevent

* define httpevent_core and add fix eventgen-httpeventout log handler

* restore samples file

* restore unit test file

* Add license credits (#222)

* Feature/multi indexes (#224)

* Fix jinja template bug under SA-Eventgen app

* Update docs (#146)

* Updated docs, added release notes to changelog

* Bump version

* add python code style lint and format

* Add more unit tests for config module

* Pep8 (#151)

* Format to code standards, addressed linter errors/warnings

* add extendIndexes feature

* set extendIndexes as a list value

* upate doc, change num to weight

* calculate generate rate and use sequential index replacement

* update dockerfile

* randomize index replacement in each batch

* Fix jinja template bug under SA-Eventgen app

* Update docs (#146)

* Updated docs, added release notes to changelog

* add python code style lint and format

* Add more unit tests for config module

* calculate generate rate and use sequential index replacement

* update dockerfile

* randomize index replacement in each batch

* fix no len dict out of range

* clean duplicate code

* Revert "Metrics output plugin" (#226)

* [issue 217]disable logging queue in multiprocess mode (#223)

* disable logging queue in multiprocess mode

* Fixed fileName (#229)

* Issue 201 (#221)

* Removing unavailable servers

* Removed nonresponding httpserver

* Added a counter

* fixed a bug

* Added docs and spec

* Addressing unused var

* fixing url reference

* fixed currentreadsize

* Httpevent str formatting

* fix #166 (#192)

* Change release version to 6.4.0 and add release notes (#232)

* fix modinput init error (#234)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Malfunctioning Eventgen due to potential bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants