How To Automate UI Testing

Most software has a user interface.  That means that most test teams spend time testing that interface.  The easiest way to do this is to just click on all the buttons and make sure the right thing happens.  This works, but it doesn't scale.  Eventually you look at the hordes of testers clicking buttons and decide that there has to be a better way.  Enter UI automation.  There are plenty of tools and toolkits to automate UI.  These range from tools like VisualTest to toolkits like Microsoft UI Automation for WPF.  These tools make automating the interface easy...for a while.  After some time you realize that you are spending all of your time updating the test scripts to reflect UI changes and very little time actually testing the product.

The difficulty is that UI is easy to change and the changes often come very late in the product.  This means that the UI is constantly changing.  For every change of the UI, the tests have to be updated.  The truth is that poking the UI buttons--either manually or automatically--doesn't really work.  They are both leave you running on a treadmill doing the same thing over and over and over.  Is there a better way?  I believe that there is.

The better way is to not test the UI.  At least, don't test it until very late.  Instead, test the functionality behind the UI.  In truth, we usually don't test the UI to make sure the button works, we test the UI to make sure the functionality represented by the button works.  For instance, in a word processor, we don't click the bold button to make sure that the button works so much as to check that text is actually turned bold when such an action is required.  It would be better if we could test the ability to make text bold without having to worry about navigating through the UI buttons to make it happen.  The MakeTextBold function is not likely to change its interface very often whereas the button is prone to changing shape, moving around, becoming a menu item, etc.  If we were testing MakeTextBold instead of the bold button, we wouldn't have to patch our test code every time the designers got a new idea.

This is all well and good but it doesn't work for most software products.  The trouble is the way that they are written.  UI toolkits encourage developers to write the code invoked by the button click event right in the button click handler.  The code might look something like this:

BoldButtonHandler(Context context)

{

    Region selectedText = Framework.GetSelectedRegion();

    selectedText.Font = selectedText.Font.Bold();

}

In reality it would be a lot more complicated, but I think you can get the general idea from this.  The difficulty with this code is that the only way to invoke it is to invoke a button click.  That means finding the right button in the UI and clicking on it.

We can, however, restructure our code.  Instead of doing the work implied by the button click in the button handler, we can make the button handler a very thin layer that just calls another function to do the work.  The advantage is that we decouple the button clicking from the work it implies.  We can then test the work unit without worrying about where the buttons are in the UI.  The new code might look like this:

BoldButtonHandler(Context context)

{

    MakeTextBold();

}

MakeTextBold()

{

    Region selectedText = Framework.GetSelectedRegion();

    selectedText.Font = selectedText.Font.Bold();

}

This requires either a lot of rewriting or, better still, starting from the beginning of the project with the right patterns.  One advantage is that testing your application becomes resilient to changes in the UI.  Another advantage is that you can now write unit tests for the UI-based functionality.  Should you desire to expose your application's functionality for scripted automation, this also becomes a lot easier.  It's definitely worth the extra effort to go this route.

What about testing the actual UI?  Surely we still need to make sure that the bold button actually makes text bold.  We do.  But we can wait until really late in the process after the UI is stabilized to do this.  We know all of the functionality works correctly and so we can wait to make sure the buttons works.  This split-approach to testing the UI also provides the benefit that when something breaks in the UI test, you know it is in the UI code and not in the underlying functional code.

Do you have additional techniques or methods to test UI?  If so, I'd love to hear them.