AWET (Automated Web application testing based on Exploratory Testing) is a tool that takes a set of recorded test cases as input to generate further test cases for Web applications. Test cases should be provided in the form of JUnit tests that can be executed by Selenium. AWET crawls the application to explore it and to create a test model for generating test cases automatically. It exploits the input test cases to steer this process rather than just randomly crawling the application.
AWET uses Selenium as well as an extended version of Crawljax.
The runtime platform and the experimental setup are adopted from another open source project, DANTE. Some of the setup instructions that are listed below are the same as those listed for the DANTE project. Nevertheless, we share (repeat) these instructions together with those specific for AWET for the sake of completeness.
Note that the generated test cases depend on the set of recorded test cases, which are provided as input. This repository also includes a set of sample test cases recorded for 5 Web applications and the corresponding results.
A virtual machine running Ubuntu 18.04 LTS is available for download here . The virtual machine contains this repository and all the required dependencies.
The virtual machine is exported in the .ova format, a platform-independent distribution format for virtual machines. Instructions on how to import an .ova format virtual machine in VirtualBox are listed below:
https://www.techjunkie.com/ova-virtualbox/
The project should be cloned to the workspace folder of virtual box and absolute paths in the docker scripts must be updated. For instance, the following path variable must be updated to locate the input test cases:
private static final String BASE_FILE_FOLDER_PATH = "/home/nezihsunman/Desktop/Tests";
The script to be used for starting the crawling process is run-crawling.sh.
The first argument is the application_name. The available values are:
phoenix|dimeshift|splittypie|retroboard|petclinic
The second argument is the headless flag which determines if the browser starts headless or with the GUI. The available values are:
true|false
The third argument is a number, crawling_max_runtime which determines a timeout for the crawler, e.g.,
./run-crawling.sh dimeshift false 5
After the crawling the folder awet/applications/dimeshift/localhost/crawl-with-inputs is created, which contains the results of the crawling. In the folder awet/applications/dimeshift there is a file called selenium-actions-dimeshift-fired.txt which lists the test cases created by the crawler while it was exploring the application.
In this step the awet/applications/dimeshift/seletion-actions-dimeshift-fired.txt is used for generating an executable test suite. The script to create the test suite and run it is generate-java-project-from-crawling.sh.
The first argument is application_name and the second argument is headless. Following with the dimeshift example the command to run is:
./generate-java-project-from-crawling.sh dimeshift false
The command generates the Java project with the test suite and runs it. It also collects the coverage of all tests.
However, there may still be errors in run time due to the system's own dynamics. This is due to the change in the codes of the Java project according to the environment used.
In the running example with the dimeshift application, the test suite is not flaky. However, if there exist flakiness, AWET does not fix it automatically. Therefore, the generated test suite might need to be fixed. In particular, the applications splittypie and phoenix are challenging in terms of flakiness since in splittypie there are many notifications that hide Web elements and the menu in phoenix is not deterministic (the order of the items in the menu keeps changing).
Another task that has to be performed manually is the implementation of the reset state class, which is called both by the validation script (i.e. tedd) and when the tests are run in isolation. An example is the ResetAppState class of the dimeshift application.
However, with the dimeshift application, it is not needed to implement the reset state class, since closing the browser after each test case execution is enough to completely clean up the state of the Web application.
The script to execute the tests and measure coverage is run-tests-and-measure-coverage.sh. The first argument is the application_name, the second argument is the headless flag and the third argument is the tool like dante and crawljax. The fourth argument is the extraction_strategy but it is required only when the tool is dante.
AWET employs an extended version of Crawljax. Hence, the headless and tool arguments should be set as true and crawljax, respectively for using AWET.
To run the tests and measure coverage for the dimeshift application we execute the following command for instance:
./run-tests-and-measure-coverage.sh dimeshift true crawljax
The log files logs_RunTests_[technique]_dimeshift.txt in the Desktop folder report the tests that passed as well as the tests that broke. The log files logs_MeasureCoverageOfTests_[technique]_dimeshift.txt in the Desktop folder report the code coverage achieved by the non-broken tests.