Test Driving User Interfaces

Friday, 24 October 2014

User interfaces are generally considered one of the harder things to develop with Test-Driven Development (TDD). So much so that a common recommendation is to make your user interface code as "thin" as possible, and test the layer behind the UI rather than the UI itself.

This is perfectly sound advice if you truly can get your thin UI layer so simple that it couldn't possibly break. It's also great if it means that code is being tested that previously wouldn't have been — any testing is better than no testing. However, if your UI layer is more than just a few simple controls with minimal behaviour then doing this properly requires that the "thin" UI layer actually ends up quite complex, as it has to pass through all the events generated by the UI, as well as provide facilities for any UI changes made through the UI API. At this point, testing behind the UI leaves quite a lot of complex code untested, and thus a prime breeding ground for bugs.

UI Testing with External Tools

One way to test the UI is to drive it with external testing tools. There are plenty of these around --- Wikipedia has a whole page of GUI testing tools like Rational Robot and Selenium.

In my experience, these tools are great for acceptance tests and testing the whole application through the UI to ensure that everything ties together correctly. More teams should use tools like this rather than testing manually, reserving the skill of their human tests for finding bugs in the edge cases rather than mindlessly clicking through the test script to ensure that the code works in the precise way defined by the test. That's what these tools are good at, so use them.

However, they don't really work for TDD precisely because they are external tools that drive the UI. For test-driving the UI code we need to be able to isolate just the UI layer, and ensure that it sends the right commands to the rest of the code, and is updated correctly when the rest of the code calls the provided API functions.

Test-driving the UI Layer

The best way to test drive the UI layer is to drive it from a test function written in the same language. If you're test-driving a JavaScript UI, your tests should be in JavaScript; if you're test-driving a C++ UI, your tests should be in C++.

You can use a test framework, but you don't have to. I generally find that the tests are easier to read and write if you use a framework, but for getting started it can be easier just to write some tests without a framework. The hard part is actually designing your code to support testing in this way: you need to introduce a seam between the UI code and the rest of the application, so that you can intercept calls to the backend in the tests. This is good software engineering anyway — cleanly separating concerns so the UI does UI stuff and only UI stuff — but it's not always the easiest direction to go at first.

A JavaScript Example

Suppose I'm testing a web app written in JavaScript using JQuery. Part of the app does some form of AJAX-based search — the user enters a search term in an edit box and clicks "search", and the app then does a search in the background and displays the results.

Here's our minimal HTML fragment:

    <form class="search-form">
    <p><label for="search-term">Search Term:</label>
        <input name="search-term" id="search-term" type="text"></p>
    <button class="search-submit">Search</button>
    <h3>Results</h3>
    <div class="results">
    </div>
    </form>

For testing this, we need two things:

firstly, we need to be able to create the HTML fragment in our test, so we don't need to have a whole page dedicated for each test, and
secondly, we need to be able to trap the AJAX call: we're testing the JavaScript, not the full stack.

The first requirement means that we need a way of attaching our handlers to the HTML at runtime; we can't just attach them manually in the $(document).ready() handler.

The second requirement means that the code under test need to call a function we supply to do the AJAX call, rather than calling $.ajax() or one of the helpers like $.post() or $.get().

Explicitly separating things out like this can be hard at first, but it does make things easier in the long term.

A first test: If we don't click, there's no AJAX request

So, let's write our first test: . Firstly, let's define the HTML snippet for our form:

    var search_form='<form class="search-form">'+
      '<p><label for="search-term">Search Term:</label>'+
      '<input name="search-term" id="search-term" type="text"></p>'+
      '<button class="search-submit">Search</button>'+
      '<h3>Results</h3>'+
      '<div class="results">'+
      '</div>'+
      '</form>';

Ideally, we'd like to load this from the same place it is defined on our website so it is kept up-to-date when we modify the form. I use a "fragments" directory for this sort of thing, and the main page is then assembled from the fragments in PHP. For now, we can define it directly in the test script.

Now, we define a simple test function. First, we clear out the body of the web page, and add the form. Then we create a dummy AJAX post function that just records the supplied data, and pass it to our form handler creation function.

This is all just setup, the test itself comes next: we check that our dummy function did not record an entry (no request made), and show an alert if it did.

    function test_when_search_button_is_not_clicked_ajax_request_not_sent(){
        var body=$('body');
        body.empty();
        body.append(search_form);
        var form=body.find('form');
        var posted_ajax=[];
        var post_ajax=function(url,data,handler){
            posted_ajax.push({url:url,data:data,handler:handler});
        }
        setup_search_form(form,post_ajax);
        if(posted_ajax.length !=0){
            alert("Bogus AJAX Posted");
            return false;
        }
        return true;
    }

We then need to a minimal page that loads JQuery, loads our tests, and then runs our function when the page is loaded, displaying "Success" if the test succeeded.

    <html>
    <script type="text/javascript" src="http://code.jquery.com/jquery-1.11.1.min.js"></script>
    <script type="text/javascript" src="tddui.js"></script>

    <script type="text/javascript">
        $(document).ready(function(){
            if(test_when_search_button_is_not_clicked_ajax_request_not_sent()){
                alert("Success");
            });
    </script>
    </html>

If you load this page then all you'll see is the search form; you won't see either alert. The code will fail because setup_search_form is not defined, which will show up as an error in your browser's error log. In Firefox with Firebug I get:

ReferenceError: setup_search_form is not defined
    setup_search_form(form,post_ajax);

Let's define a minimal setup_search_form function that does nothing, just so it all runs:

    function setup_search_form(form,ajax){}

Now if you refresh the page then you get a nice "Success" alert.

This test didn't do much, but we've now got our code set up it's easy to add a second test. Let's test some actual behaviour.

A second test: Clicking sends an AJAX request

OK, so no AJAX request is sent when you don't click. Not exactly rocket science, but it gets us a framework for our tests. Let's add some behaviour: when the button is clicked, then the code should send an AJAX request.

Our second test is almost identical to the first. The key part here is what we do after setting up the form. We're going to click on the button, and the default action for a form button is to submit the form, so we first set the target to # so we don't navigate off the page. Then we use JQuery to click the button, and check that the AJAX was actually posted:

    function test_when_search_button_is_clicked_ajax_request_sent(){
        var body=$('body');
        body.empty();
        body.append(search_form);
        var form=body.find('form');
        var posted_ajax=[];
        var post_ajax=function(url,data,handler){
            posted_ajax.push({url:url,data:data,handler:handler});
        }
        setup_search_form(form,post_ajax);
        form.attr('action','#');
        form.find('button').click();
        if(posted_ajax.length !=1){
            alert("No AJAX Posted");
            return false;
        }
        return true;
    }

We then need to run our new test too, so update the driver page:

        $(document).ready(function(){
            if(test_when_search_button_is_not_clicked_ajax_request_not_sent() &&
               test_when_search_button_is_clicked_ajax_request_sent())
                alert("Success");
            });

If you now load the test page then you'll see the "No AJAX Posted" alert. Let's fix the test with the simplest possible click handler:

    function setup_search_form(form,ajax){
        form.find('button').click(function(){
            ajax('',{},function(){});
            return false;
        });
    }

Refreshing the page will now get us back to the "Success" message.

It'd be nice to clear up the duplication between our tests, but first, let's get this test finished.

Finishing the second test: checking the AJAX request is correct

All we've checked so far is that an AJAX request is sent. We actually need to check that the right AJAX request is sent, so let's do that. Add some more checks after the first one:

        if(posted_ajax[0].url!='/ajax.php'){
            alert("Wrong AJAX URL");
            return false;
        }
        if(posted_ajax[0].data.request!='search'){
            alert("AJAX request is wrong");
            return false;
        }

If you refresh the test page, then you'll find that you now get an alert saying that the URL is wrong. If you fix that, then you'll get an alert complaining about the request. Let's fix both:

    function setup_search_form(form,ajax){
        form.find('button').click(function(){
            ajax('/ajax.php',{request:'search'},function(){});
            return false;
        });
    }

This gets us back to our nice "Success" alert. Let's now clean up that duplication.

Removing duplication between tests

These tests share a common setup, so let's refactor to extract that and simplify the code:

    function setup_search_test(){
        var test_data={};
        test_data.body=$('body');
        test_data.body.empty();
        test_data.body.append(search_form);
        test_data.form=test_data.body.find('form');
        test_data.posted_ajax=[];
        test_data.post_ajax=function(url,data,handler){
            test_data.posted_ajax.push({url:url,data:data,handler:handler});
        }
        setup_search_form(test_data.form,test_data.post_ajax);
        return test_data;
    }

    function test_when_search_button_is_not_clicked_ajax_request_not_sent(){
        var test_data=setup_search_test();
        if(test_data.posted_ajax.length !=0){
            alert("Bogus AJAX Posted");
            return false;
        }
        return true;
    }

    function test_when_search_button_is_clicked_ajax_request_sent(){
        var test_data=setup_search_test();
        test_data.form.attr('action','#');
        test_data.form.find('button').click();
        if(test_data.posted_ajax.length !=1){
            alert("No AJAX Posted");
            return false;
        }
        if(test_data.posted_ajax[0].url!='/ajax.php'){
            alert("Wrong AJAX URL");
            return false;
        }
        if(test_data.posted_ajax[0].data.request!='search'){
            alert("AJAX request is wrong");
            return false;
        }
        return true;
    }

We can verify that everything is still working by refreshing our test page: we still get the "Success" alert, so no problems.

Now let's add some more behaviour.

A third test: Extracting data from the UI

For our next test, let's do a bit more work with the UI. It's all very well having the search button send an AJAX request, but we want to actually search for the supplied term, so let's do that. Here's our new test:

    function test_when_search_button_is_clicked_search_term_in_ajax(){
        var test_data=setup_search_test();
        test_data.form.attr('action','#');
        var search_term="green widgets";
        test_data.form.find('#search-term').val(search_term);
        test_data.form.find('button').click();
        if(test_data.posted_ajax.length !=1){
            alert("No AJAX Posted");
            return false;
        }
        if(test_data.posted_ajax[0].data.term!=search_term){
            alert("AJAX search term is wrong");
            return false;
        }
        return true;
    }

And here's our updated driver code:

        $(document).ready(function(){
            if(test_when_search_button_is_not_clicked_ajax_request_not_sent() &&
               test_when_search_button_is_clicked_ajax_request_sent() &&
               test_when_search_button_is_clicked_search_term_in_ajax())
                alert("Success");
            });

If you refresh the page now you'll see the "search term is wrong" error message. You should also see our search term ("green widgets") in the search box. Let's fix the error:

    function setup_search_form(form,ajax){
        form.find('button').click(function(){
            ajax('/ajax.php',
                 {request:'search',
                  term:form.find('#search-term').val()},
                 function(){});
            return false;
        });
    }

Which brings us back to our "Success" alert.

OK, so that's a lot of test code for a simple function, but we know that if we change it in a way that affects something then we'll know, and we're still completely separate from the backend code.

Let's add some UI updates for while we're waiting for the result.

Test four: Updating the UI

Our first few tests have been focused on getting the AJAX request right. However, we want the user to know that something is happening when they make their request, so let's handle that. If the user clicks the search button, both it and the search term box should be disabled, and the results block should show a "searching for ..." message.

    function test_when_search_button_is_clicked_UI_updated_to_show_searching(){
        var test_data=setup_search_test();
        test_data.form.attr('action','#');
        var search_term="red widgets";
        test_data.form.find('#search-term').val(search_term);
        test_data.form.find('button').click();
        if(!test_data.form.find('button').prop("disabled") ||
           !test_data.form.find('#search-term').prop("disabled")){
            alert("UI not disabled");
            return false;
        }
        if(test_data.form.find('.results').text()!="Searching for "+search_term){
            alert("Results field has wrong content");
            return false;
        }
        return true;
    }

If we add that test to our driver code, then we'll get an alert complaining about the UI not being disabled. Easily fixed:

    function setup_search_form(form,ajax){
        form.find('button').click(function(){
            $(this).prop("disabled",true);
            var term_field=form.find('#search-term');
            var term=term_field.val();
            term_field.prop("disabled",true);
            form.find('.results').text("Searching for "+term);
            ajax('/ajax.php',
                 {request:'search',
                  term:term},
                 function(){});
            return false;
        });
    }

In a real web app, you might add some form of animation, but for now this will do. A more important feature is actually displaying the results when they come back. But first: more duplication.

Eliminating more duplication

Almost all the tests clear the form action field because they click on the button. Let's move that into the setup function:

    function setup_search_test(){
        var test_data={};
        test_data.body=$('body');
        test_data.body.empty();
        test_data.body.append(search_form);
        test_data.form=test_data.body.find('form');
        test_data.form.attr('action','#');
        test_data.posted_ajax=[];
        test_data.post_ajax=function(url,data,handler){
            test_data.posted_ajax.push({url:url,data:data,handler:handler});
        }
        setup_search_form(test_data.form,test_data.post_ajax);
        return test_data;
    }

Now on to test five.

Test five: The results are in!

Way back at test one we allowed the caller to supply a handler to the ajax call, which we duly recorded, but haven't used for anything. Now it's time to use it: we can call it from the test to indicate that the results of the AJAX call are back.

The set up is similar to what we've done before: enter a search time and click search:

    function test_results_of_search_go_in_results_div(){
        var test_data=setup_search_test();
        var search_term="red widgets";
        test_data.form.find('#search-term').val(search_term);
        test_data.form.find('button').click();
        if(test_data.posted_ajax.length !=1){
            alert("No AJAX Posted");
            return false;
        }

Now we need some results to pass to the handler:

        var result_data={
            results:[
                "red spinning widgets",
                "fast red widgets",
                "big red widgets"
            ]
        };

        test_data.posted_ajax[0].handler(result_data);

And then we check the results. In this case, we're verifying that the results are stored in a <UL> tag that is the sole element in the results block. The final check for "spurious text" ensures that we've removed the "searching for" text we added previously.

        var result_div=test_data.form.find('.results');
        if(result_div.children().length!=1){
            alert("Should be exactly one child in result div");
            return false;
        }
        if(result_div.find('ul').length!=1){
            alert("Results are an unordered list");
            return false;
        }
        var list_entries=result_div.find('ul li');
        if(list_entries.length!=result_data.results.length){
            alert("One list element per result entry");
            return false;
        }
        for(var i=0;i<list_entries.length;++i){
            var entry=$(list_entries[i]);
            if(entry.text()!=result_data.results[i]){
                alert("Result entry " + i + " is wrong");
                return false;
            }
        }
        if(result_div.text() != result_div.find('ul').text()){
            alert("Spurious text");
            return false;
        }

        return true;
    }

If you add the test to the driver page then it will fail: the results block has no children until we add some.

Let's make it pass by implementing the handler function:

    function setup_search_form(form,ajax){
        var results_field=form.find('.results');
        var handle_results=function(data){
            var result_list=$('<ul></ul>');
            for(var i=0;i<data.results.length;++i){
                var entry=$('<li>');
                entry.text(data.results[i]);
                result_list.append(entry);
            }
            results_field.empty();
            results_field.append(result_list);
        };

The rest is pretty much as before, except we pass in our new handler function to the ajax call:

        form.find('button').click(function(){
            $(this).prop("disabled",true);
            var term_field=form.find('#search-term');
            var term=term_field.val();
            term_field.prop("disabled",true);
            results_field.text("Searching for "+term);
            ajax('/ajax.php',
                 {request:'search',
                  term:term},
                 handle_results);
            return false;
        });
    }

And we're back at "Success".

The search button and search term box are still disabled though, so let's fix that.

Test six: Re-enabling form fields

When the results come back, we want our form fields to be re-enabled. That's easy to test for:

    function test_when_results_back_enable_fields(){
        var test_data=setup_search_test();
        var search_term="red widgets";
        test_data.form.find('#search-term').val(search_term);
        test_data.form.find('button').click();
        if(test_data.posted_ajax.length !=1){
            alert("No AJAX Posted");
            return false;
        }
        var result_data={
            results:[
                "red spinning widgets",
                "fast red widgets",
                "big red widgets"
            ]
        };

        test_data.posted_ajax[0].handler(result_data);
        if(test_data.form.find('button').prop("disabled") ||
           test_data.form.find('#search-term').prop("disabled")){
            alert("UI not re-enabled");
            return false;
        }

        return true;
    }

Add to the driver, and refresh to check to you get the "UI not re-enabled" message, and then we can fix it: update the ajax result handler to enable the controls.

    function setup_search_form(form,ajax){
        var term_field=form.find('#search-term');
        var results_field=form.find('.results');
        var submit_button=form.find('button');
        var handle_results=function(data){
            var result_list=$('<ul></ul>');
            for(var i=0;i<data.results.length;++i){
                var entry=$('<li>');
                entry.text(data.results[i]);
                result_list.append(entry);
            }
            results_field.empty();
            results_field.append(result_list);
            submit_button.prop("disabled",false);
            term_field.prop("disabled",false);
        };

        submit_button.click(function(){
            $(this).prop("disabled",true);
            var term=term_field.val();
            term_field.prop("disabled",true);
            results_field.text("Searching for "+term);
            ajax('/ajax.php',
                 {request:'search',
                  term:term},
                 handle_results);
            return false;
        });
    }

Which brings us back to "Success".

What we haven't yet handled is what to do when the AJAX call fails. This is where separating the UI from the back-end code really helps us out — it's exceedingly hard to engineer failure conditions when you're doing whole-system testing through the UI, but we can just trigger failure because we feel like it. So, let's do it.

Test seven: Failing AJAX calls

In order to simulate failure, we need a failure handler for our AJAX calls. So let's add a parameter for handling failures to our dummy AJAX function:

        test_data.post_ajax=function(url,data,handler,failure_handler){
            test_data.posted_ajax.push(
                {url:url,data:data,handler:handler,failure:failure_handler});
        }

In the test, rather than supplying results, we can invoke the failure handler. That's easily done, but what do we want the result to be?

The easiest thing for now is to put some form of error status in the results block, and re-enable the search controls, so let's do that:

    function test_ajax_failure(){
        var test_data=setup_search_test();
        var search_term="red widgets";
        test_data.form.find('#search-term').val(search_term);
        test_data.form.find('button').click();
        if(test_data.posted_ajax.length !=1){
            alert("No AJAX Posted");
            return false;
        }
        if(!test_data.posted_ajax[0].failure){
            alert("No failure handler specified");
            return false;
        }
        test_data.posted_ajax[0].failure(404,"Not Found","");
        var result_div=test_data.form.find('.results');
        if(result_div.text()!="Unable to retrieve search results: error 404 (Not Found)"){
            alert("Result text is wrong");
            return false;
        }
        if(test_data.form.find('button').prop("disabled") ||
           test_data.form.find('#search-term').prop("disabled")){
            alert("UI not re-enabled");
            return false;
        }
        return true;
    }

Adding this to our test driver should give us a "no failure handler" error. This is easily fixed by adding a handler to our form setup:

        var handle_failure=function(status,error_text,response_data){
            results_field.empty();
            results_field.text(
                "Unable to retrieve search results: error " + status + " ("+error_text+")");
            submit_button.prop("disabled",false);
            term_field.prop("disabled",false);
        }

        submit_button.click(function(){
            $(this).prop("disabled",true);
            var term=term_field.val();
            term_field.prop("disabled",true);
            results_field.text("Searching for "+term);
            ajax('/ajax.php',
                 {request:'search',
                  term:term},
                 handle_results,
                 handle_failure);
            return false;
        });

Which brings us back to the familiar "Success" message.

I'll leave the example there. If this was part of a real web app then there's lots more that would need to be done, along with corresponding tests, but for our simple example this will suffice. I hope you can see how this could be extended to test other scenarios.

Check out the final driver page and JavaScript code for this example.

A real `ajax` function for this code

This code relies on an ajax function to request data from the server, which we have mocked out in the tests. Here is a simple implementation that uses JQuery's post() function:

    function jquery_ajax(url,data,handler,failure_handler){
        $.post(url,data,handler).fail(function(xhr){
            if(failure_handler){
                failure_handler(xhr.status,xhr.statusText,xhr.responseText);
            }
        });
    }

This could then be passed to our setup_search_form function in live code to make real AJAX requests.

Test frameworks

This example doesn't use any external code except JQuery, just to show how easy it is to get started, but there are plenty of test frameworks available that make it easier to write tests, or view the results. Personally, I like QUnit for JavaScript, but use whatever takes your fancy. A test framework will generally record how many of your tests passed or failed, rather than using alert() as we have here. They also tend to offer various checks like assertEquals(), or assertLessThan() which will record the supplied parameters as well as marking the test fail. This can make it easier to work out what went wrong if a test fails unexpectedly.

Other languages

This example was JavaScript, but the overall idea is the same in whatever language you use. Most GUI frameworks provide an API for querying the state of the UI, and can also be made to trigger events as-if a user has made an action. For example, when testing Windows applications in this way you can call SendMessage and PostMessage from within the tests to simulate the messages sent by the system when the user interacts with the application via the mouse or keyboard.

End note

As you've seen from this example, test-driving UIs is possible. It's still a good idea to make the UI layer as thin as possible, but that's just general good software engineering. Indeed, test-driving the UI can actually reduce coupling by forcing you to introduce an interface where previously you might have used another subsystem directly.

Posted by Anthony Williams
[/ tdd /] permanent link
Tags: tdd, testing, user interface design
Stumble It! | Submit to Reddit | Submit to DZone

Comment on this post

If you liked this post, why not subscribe to the RSS feed or Follow me on Twitter? You can also subscribe to this blog by email using the form on the left.

Just Software Solutions

About Us

Technical Writings

Subscribe to Blog

Blog Archives

Test Driving User Interfaces

Friday, 24 October 2014

UI Testing with External Tools

Test-driving the UI Layer

A JavaScript Example

A first test: If we don't click, there's no AJAX request

A second test: Clicking sends an AJAX request

Finishing the second test: checking the AJAX request is correct

Removing duplication between tests

A third test: Extracting data from the UI

Test four: Updating the UI

Eliminating more duplication

Test five: The results are in!

Test six: Re-enabling form fields

Test seven: Failing AJAX calls

A real `ajax` function for this code

Test frameworks

Other languages

End note

No Comments

Add your comment

Just Software Solutions

About Us

Technical Writings

Subscribe to Blog

Blog Archives

Test Driving User Interfaces

Friday, 24 October 2014

UI Testing with External Tools

Test-driving the UI Layer

A JavaScript Example

Results

A first test: If we don't click, there's no AJAX request

A second test: Clicking sends an AJAX request

Finishing the second test: checking the AJAX request is correct

Removing duplication between tests

A third test: Extracting data from the UI

Test four: Updating the UI

Eliminating more duplication

Test five: The results are in!

Test six: Re-enabling form fields

Test seven: Failing AJAX calls

A real ajax function for this code

Test frameworks

Other languages

End note

No Comments

Add your comment

A real `ajax` function for this code