Two typical real life test suites

Automated functional testing

This automated test tool is most beneficial for functional testing. “Functional” in this context means that you test core/all features of your application for whether they fulfil the end-user specification.

This often means that you are not just testing one single web page at a time, but a whole workflow of different web pages in interaction between different end users.

This is how a small “typical” functional test suite could look like:

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite>
    <inherit>
        <baseurl>http://automateyourtests.com/demo_application</baseurl>
        <status>200</status>
        <status>301</status>
        <status>302</status>
        <notstatus>404</notstatus>
        <notstatus>500</notstatus>
        <notcontains><![CDATA[<b>Fatal error</b>:]]></notcontains>
        <notcontains><![CDATA[<b>Notice</b>:]]></notcontains>
        <notcontains><![CDATA[<b>Parse error</b>:]]></notcontains>
        <notcontains><![CDATA[<b>Warning</b>:]]></notcontains>
    </inherit>
    <setup>
        Reset DB
    </setup>
    <logins title="Let each test step execute in the context of an authenticated user" cookiedir="{TEMPDIR}">
        <user label="Homer Simpson">
            <url>/wp-login.php?testcookie=1</url>
            <post><![CDATA[log=homer&pwd=doh]]></post>
            <notcontains><![CDATA[<div id="login_error">]]></notcontains>
        </user>
    </logins>
    <fixture>
        <setup label="Reset DB"><![CDATA[mysql --host={YOUR_DB_HOST} --user={YOUR_DB_USER} --password={YOUR_DB_PW} {YOUR_DB_NAME} < {YOUR_DB_SQL_PATH}.sql]]></setup>
    </fixture>
    <userstory title="Test that the comments from the JSON data feed match up with the comments shown on the web site">
        <scenario title="Test that the existing test database comment matches up">
            <step title="The test database comment appears in the comment stream">
                <url>/uncategorized/hello-world</url>
                <contains>Test Comment</contains>
            </step>
            <step title="The test database comment also appears in the JSON data feed">
                <url>/?json=get_posts</url>
                <xpath>//content[contains(text(),"Test Comment")]</xpath>
                <xpath>//status[.='ok']</xpath>
            </step>
            <step title="The JSON data feed has a comment count of 1">
                <url>/?json=get_posts</url>
                <xpath>//comment_count[.=1]</xpath>
                <xpath>//status[.='ok']</xpath>
            </step>
        </scenario>
        <scenario title="Test that a public user comment matches up">
            <step title="A public user posts a comment">
                <url>/wp-comments-post.php</url>
                <post><![CDATA[comment=Public+User+Comment&comment_post_ID=1]]></post>
            </step>
            <step title="The public user comment appears in the comment stream">
                <url>/uncategorized/hello-world</url>
                <contains>Public User Comment</contains>
            </step>
            <step title="The public user comment also appears in the JSON data feed">
                <url>/?json=get_posts</url>
                <xpath>//content[contains(text(),"Public User Comment")]</xpath>
                <xpath>//status[.='ok']</xpath>
            </step>
            <step title="The JSON data feed has a comment count of 2">
                <url>/?json=get_posts</url>
                <xpath>//comment_count[.=2]</xpath>
                <xpath>//status[.='ok']</xpath>
            </step>
        </scenario>
        <scenario title="Test that a logged-in user comment matches up">
            <step title="The JSON data feed STILL has a comment count of 1 (there already is a test comment in the database)">
                <url>/?json=get_posts</url>
                <xpath>//comment_count[.=3]</xpath>
                <xpath>//status[.='ok']</xpath>
            </step>
            <step title="A logged-in user posts a comment">
                <user>Homer Simpson</user>
                <url>/wp-comments-post.php</url>
                <post><![CDATA[comment=Logged+In+User+Comment&comment_post_ID=1]]></post>
            </step>
            <step title="The posted comment appears in the comment stream">
                <url>/uncategorized/hello-world</url>
                <contains>Logged In User Comment</contains>
            </step>
            <step title="The posted comment also appears in the JSON data feed">
                <url>/?json=get_posts</url>
                <xpath>//content[contains(text(),"Logged In User Comment")]</xpath>
                <xpath>//status[.='ok']</xpath>
            </step>
            <step title="The JSON data feed NOW has a comment count of 3">
                <url>/?json=get_posts</url>
                <xpath>//comment_count[.=3]</xpath>
                <xpath>//status[.='ok']</xpath>
            </step>
        </scenario>
    </userstory>
</testsuite>

Automated technical testing

“Technical” in this context means that you interrogate a URL or web form with multiple use cases of edge case data.

Whilst such a black-box testing approach can not fully replace unit testing for your web application, the following test suite is much, much quicker to write.

This is how a small “typical” technical test suite could look like:

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite>
    <inherit>
        <baseurl>http://automateyourtests.com/demo_application</baseurl>
        <status>200</status>
        <status>301</status>
        <status>302</status>
        <notstatus>404</notstatus>
        <notstatus>500</notstatus>
        <notcontains><![CDATA[<b>Fatal error</b>:]]></notcontains>
        <notcontains><![CDATA[<b>Notice</b>:]]></notcontains>
        <notcontains><![CDATA[<b>Parse error</b>:]]></notcontains>
        <notcontains><![CDATA[<b>Warning</b>:]]></notcontains>
    </inherit>
    <fixture>
        <setup label="Reset DB"><![CDATA[mysql --host={YOUR_DB_HOST} --user={YOUR_DB_USER} --password={YOUR_DB_PW} {YOUR_DB_NAME} < {YOUR_DB_SQL_PATH}.sql]]></setup>
    </fixture>
    <setup>
        Reset DB
    </setup>
    <userstory title="Test that the Wordpress login form correctly allows/denies correct/incorrect username and password combinations">
        <scenario title="Test that a wrong username and password combination DENIES access">
            <step title="Go to login page">
                <url>/wp-login.php</url>
            </step>
            <step title="Login with WRONG username and WRONG password">
                <url>/wp-login.php?testcookie=1</url>
                <post><![CDATA[log=WRONG_USERNAME&pwd=WRONG_PASSWORD]]></post>
                <contains>Incorrect username or password</contains>
            </step>
            <step title="Login with WRONG username and CORRECT password">
                <url>/wp-login.php?testcookie=1</url>
                <post><![CDATA[log=WRONG_USERNAME&pwd=lucy]]></post>
                <contains>Incorrect username or password</contains>
            </step>
            <step title="Login with CORRECT username and WRONG password">
                <url>/wp-login.php?testcookie=1</url>
                <post><![CDATA[log=charlie&pwd=WRONG_PASSWORD]]></post>
                <contains>Incorrect username or password</contains>
            </step>
        </scenario>
        <scenario title="Test that a correct username and password combination ALLOWS access">
            <step title="Go to login page">
                <url>/wp-login.php</url>
            </step>
            <step title="Login with CORRECT username and CORRECT password">
                <url>/wp-login.php?testcookie=1</url>
                <post><![CDATA[log=homer&pwd=doh]]></post>
                <notcontains>Incorrect username or password</notcontains>
            </step>
        </scenario>
    </userstory>
</testsuite>