Q: Define directory to store output of flow

[thim81]

Hi @kensoh,

Again a practical question, is it possible to define a specific output folder for the snapshots, .xml, .log. js files?

Currently it is always the same folder as where the flow file is.

[kensoh]

Hi @thim81 steps such as snap, save, dump allows a parameter to specify the filename / and folder, but for the generated files .log .xml .js they are in the same folder as the source flow file. Can you share more about your use case?

I assume maybe you want to make multiple runs and retain the generated files for previous runs. At the moment it looks like either moving the files to somewhere else (eg folder with date) with a wrapper script, or copying the script to new folders or new filenames before running.

[thim81]

@kensoh

That was indeed the use case, to have a history per run.

So following your feedback I could a add var fs = require('fs'); for manipulating the files and moving them to another directory.

[kensoh]

That's an interesting way of handling it, to do it within the automation flow. As TagUI wasn't designed primarily for test automation, you have raised interesting ideas that can also improve the tool from process automation angle. Let me know if you run into issues doing it that way.

To avoid Node.js dependency, the fs above is directly inherited from PhantomJS fs and it does have an extensive list of methods for handling moving, creating directories etc. http://phantomjs.org/api/fs/

[thim81]

So let me explain shortly why I like your TagUI so much:

• it's based on all the good frameworks and unifies the way of working with them
• the syntax is so easy, clean and uniform (no JS like promise chains / no different syntax for each functionality ...)
• It can do everything I need it to do using the simple syntax: take screenshot, click, enter text, run assertion tests, ...
• It support Continuous integration workflow because of the xunit xml report
• Very easy to setup
• Easy to include dynamic data via CSV files
• It allows customizing of data through basic JS
• Support headless and browser testing for Firefox AND chrome

This all allows me to tackle the following use cases:

• replace repetitive manual testing UI project with scripted testing
• keep the learning curve for setting up and creating tests as low as possible but yet have a very flexible tool, thanks to your simple syntax and all the options
• run automated assertion tests on the testing flows with testing reports, which allows the usage of CI tools like GitLab CI

The next thing what could be interesting is to add the option to do CSS regression testing like:

• https://github.com/Huddle/PhantomCSS >> which is using the same stack
• https://github.com/BBC-News/wraith
  Which I think is possible once there is a "history" of the output. This would then allow comparison of the snaps and use this as an additional validation step in the CI workflow.

So use cases covered:

• automated behaviour testing
• automated application flow testing
• reports on the testing
• visually track test via screenshots
• automated UI change testing (* by hooking in PhantomCSS or Wraith)

[kensoh]

Thanks for your encouragement! The primary aim of the tool is not so much for test automation, but now it only has the latest execution log. I think there is benefit from having a way to record logs more than just the latest last run. This sounds like an enhancement to consider how to implement.

Is it correct to say that what you are primarily interested in is having some form of records for past executions? And specifying directory is an idea to implement, which is not as important as long as there is ability to track historical logs / screenshots?

I'm thinking of whether implementing it as part of the main runner or outside. I have some thoughts to make a CLI helper for TagUI, so that inputs / executions can be made more natural. For example, typing erina get my bank balance on the command prompt will trigger the correct automation flow and returns the answer. Ultimately having natural invocation like this can be a pre-requisite to voice control.

[kensoh]

Committed baseline option to master with following comments. Decided to implement within existing main runners framework as it can be done elegantly and easily. Instead of delaying for natural-language CLI-helper wrapper.

When baseline option is used, a folder named baseline will be created at the location of the automation flow file. A copy of the flow file + the execution log + the output files (those specified as relative paths in the flow) will be stored there instead of the original flow file location.

After thinking through, this should be a good balance between convenience to easily create baseline snapshot, and having more complicated configuration. Since baseline is the single source of truth as the benchmark, users can run with baseline option whenever they think the run results are the way they are supposed to be. Users can then compared subsequent runs with their archived/captured baseline when they need to, on errors or for other reasons.

not yet tested for windows runner. but that’s ok, shouldn’t break
existing functionality when baseline option is not used. test that when
have access to windows env. tested for macos/linux

decided to put as part of existing framework because it can be done
within the main runners tagui / tagui.cmd easily and elegantly. no need
to wait until extending to natural-language cli helper to implement.

@thim81 - I've tested on macOS / Linux but can only test the Windows runner when have access to Windows laptop. Do give a shout if you run into any issue!

[kensoh]

updated commit and tested successfully for windows