So far, you had to execute your website test suites more or less manually by executing php run.php testsuite=”your_test_suite.xml” on the command prompt.
These are some possible strategies of how you achieve full test automation of the test suites you have written:
- First amend the provided run.php to not echo $dispatcher->getLog(‘log’) and echo $dispatcher->getLog(‘summary’) to the screen anymore, but instead to save them to a log file and/or email them to your email address.
Then you’ll have the choice. You could either run your test suites as:
- Functional or regression tests on the development server: Put technical hooks in place that make it mandatory for all developers to automatically run your test suite whenever they upload new programming code to the development server. You could, for example, integrate it with your:
- Continuous Integration System (e.g. Jenkins): Run php run.php testsuite=”your_test_suite.xml” inside your Continuous Integration System and the provided exit($dispatcher->getExitCode()) will tell it, whether at least one test has FAILED (exit code 1), or all tests have PASSED (exit code 0).
- Version Control System: Run php run.php testsuite=”your_test_suite.xml” inside of one the hooks of your Version Control System and the provided exit($dispatcher->getExitCode()) will tell it, whether at least one test has FAILED (exit code 1), or all tests have PASSED (exit code 0). If you use GIT, for example I can recommend /.git/pre-push to be a suitable hook.
- Sanity tests on the live server: Anyone who manages the live version of the website application could schedule a sanity test suite (e.g. every 5 minutes or every 24 hours) using a cronjob or Windows Task Manager. In this case you would only safeguard the most crucial functions and URLs of the live environment, and obviously, you would not be able to use the test automation database reset feature. These tests are still very valuable, because they protect you not only against programming bugs, but also against live system faults of any other nature.