Automated functional testing
This automated test tool is most beneficial for functional testing. “Functional” in this context means that you test core/all features of your application for whether they fulfil the end-user specification.
This often means that you are not just testing one single web page at a time, but a whole workflow of different web pages in interaction between different end users.
This is how a small “typical” functional test suite could look like:
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite>
<inherit>
<baseurl>http://automateyourtests.com/demo_application</baseurl>
<status>200</status>
<status>301</status>
<status>302</status>
<notstatus>404</notstatus>
<notstatus>500</notstatus>
<notcontains><![CDATA[<b>Fatal error</b>:]]></notcontains>
<notcontains><![CDATA[<b>Notice</b>:]]></notcontains>
<notcontains><![CDATA[<b>Parse error</b>:]]></notcontains>
<notcontains><![CDATA[<b>Warning</b>:]]></notcontains>
</inherit>
<setup>
Reset DB
</setup>
<logins title="Let each test step execute in the context of an authenticated user" cookiedir="{TEMPDIR}">
<user label="Homer Simpson">
<url>/wp-login.php?testcookie=1</url>
<post><![CDATA[log=homer&pwd=doh]]></post>
<notcontains><![CDATA[<div id="login_error">]]></notcontains>
</user>
</logins>
<fixture>
<setup label="Reset DB"><![CDATA[mysql --host={YOUR_DB_HOST} --user={YOUR_DB_USER} --password={YOUR_DB_PW} {YOUR_DB_NAME} < {YOUR_DB_SQL_PATH}.sql]]></setup>
</fixture>
<userstory title="Test that the comments from the JSON data feed match up with the comments shown on the web site">
<scenario title="Test that the existing test database comment matches up">
<step title="The test database comment appears in the comment stream">
<url>/uncategorized/hello-world</url>
<contains>Test Comment</contains>
</step>
<step title="The test database comment also appears in the JSON data feed">
<url>/?json=get_posts</url>
<xpath>//content[contains(text(),"Test Comment")]</xpath>
<xpath>//status[.='ok']</xpath>
</step>
<step title="The JSON data feed has a comment count of 1">
<url>/?json=get_posts</url>
<xpath>//comment_count[.=1]</xpath>
<xpath>//status[.='ok']</xpath>
</step>
</scenario>
<scenario title="Test that a public user comment matches up">
<step title="A public user posts a comment">
<url>/wp-comments-post.php</url>
<post><![CDATA[comment=Public+User+Comment&comment_post_ID=1]]></post>
</step>
<step title="The public user comment appears in the comment stream">
<url>/uncategorized/hello-world</url>
<contains>Public User Comment</contains>
</step>
<step title="The public user comment also appears in the JSON data feed">
<url>/?json=get_posts</url>
<xpath>//content[contains(text(),"Public User Comment")]</xpath>
<xpath>//status[.='ok']</xpath>
</step>
<step title="The JSON data feed has a comment count of 2">
<url>/?json=get_posts</url>
<xpath>//comment_count[.=2]</xpath>
<xpath>//status[.='ok']</xpath>
</step>
</scenario>
<scenario title="Test that a logged-in user comment matches up">
<step title="The JSON data feed STILL has a comment count of 1 (there already is a test comment in the database)">
<url>/?json=get_posts</url>
<xpath>//comment_count[.=3]</xpath>
<xpath>//status[.='ok']</xpath>
</step>
<step title="A logged-in user posts a comment">
<user>Homer Simpson</user>
<url>/wp-comments-post.php</url>
<post><![CDATA[comment=Logged+In+User+Comment&comment_post_ID=1]]></post>
</step>
<step title="The posted comment appears in the comment stream">
<url>/uncategorized/hello-world</url>
<contains>Logged In User Comment</contains>
</step>
<step title="The posted comment also appears in the JSON data feed">
<url>/?json=get_posts</url>
<xpath>//content[contains(text(),"Logged In User Comment")]</xpath>
<xpath>//status[.='ok']</xpath>
</step>
<step title="The JSON data feed NOW has a comment count of 3">
<url>/?json=get_posts</url>
<xpath>//comment_count[.=3]</xpath>
<xpath>//status[.='ok']</xpath>
</step>
</scenario>
</userstory>
</testsuite>
Automated technical testing
“Technical” in this context means that you interrogate a URL or web form with multiple use cases of edge case data.
Whilst such a black-box testing approach can not fully replace unit testing for your web application, the following test suite is much, much quicker to write.
This is how a small “typical” technical test suite could look like:
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite>
<inherit>
<baseurl>http://automateyourtests.com/demo_application</baseurl>
<status>200</status>
<status>301</status>
<status>302</status>
<notstatus>404</notstatus>
<notstatus>500</notstatus>
<notcontains><![CDATA[<b>Fatal error</b>:]]></notcontains>
<notcontains><![CDATA[<b>Notice</b>:]]></notcontains>
<notcontains><![CDATA[<b>Parse error</b>:]]></notcontains>
<notcontains><![CDATA[<b>Warning</b>:]]></notcontains>
</inherit>
<fixture>
<setup label="Reset DB"><![CDATA[mysql --host={YOUR_DB_HOST} --user={YOUR_DB_USER} --password={YOUR_DB_PW} {YOUR_DB_NAME} < {YOUR_DB_SQL_PATH}.sql]]></setup>
</fixture>
<setup>
Reset DB
</setup>
<userstory title="Test that the Wordpress login form correctly allows/denies correct/incorrect username and password combinations">
<scenario title="Test that a wrong username and password combination DENIES access">
<step title="Go to login page">
<url>/wp-login.php</url>
</step>
<step title="Login with WRONG username and WRONG password">
<url>/wp-login.php?testcookie=1</url>
<post><![CDATA[log=WRONG_USERNAME&pwd=WRONG_PASSWORD]]></post>
<contains>Incorrect username or password</contains>
</step>
<step title="Login with WRONG username and CORRECT password">
<url>/wp-login.php?testcookie=1</url>
<post><![CDATA[log=WRONG_USERNAME&pwd=lucy]]></post>
<contains>Incorrect username or password</contains>
</step>
<step title="Login with CORRECT username and WRONG password">
<url>/wp-login.php?testcookie=1</url>
<post><![CDATA[log=charlie&pwd=WRONG_PASSWORD]]></post>
<contains>Incorrect username or password</contains>
</step>
</scenario>
<scenario title="Test that a correct username and password combination ALLOWS access">
<step title="Go to login page">
<url>/wp-login.php</url>
</step>
<step title="Login with CORRECT username and CORRECT password">
<url>/wp-login.php?testcookie=1</url>
<post><![CDATA[log=homer&pwd=doh]]></post>
<notcontains>Incorrect username or password</notcontains>
</step>
</scenario>
</userstory>
</testsuite>