Hiring Great Testers - Tester Roles

In my mind, there are basically three roles on a test team.  These three roles are: developers, scripters, and those who execute the test cases (runtime testers).  In reality there is a spectrum of capabilities on any team but I think most roles will be closely related to one of these three roles.

Test Developers are the heart of a modern test team.  There was a day when you could get away with hiring a few people to just use the product and call that a test team.  This is no longer the case.  Products are becoming more complex.  The lifespan of products is increasing.  More products are being created for developers instead of end users.  These have no UI to interact with so simple exploratory testing is insufficient.  To test complex products, especially over an extended lifespan, the only viable solution is test automation.  When the product is an API instead of a user interface, testing it requires programming. 

Test developers are programmers who happen to work on a test team.  It is their job to write software which executes other software and verifies the results.  The test developers need to be at least as skilled as the developers of the product.  They need to know whatever language their product is written in.  Their job is often as difficult or perhaps even more difficult than writing the product.  Sometimes the job can be done in a simple fashion where each function is just called with various parameters and a simple method of pass/fail is monitored.  This could be a success result or some notification from the program.  A better test developer will not rely on a program to tell him if the right things happened.  Instead, he will monitor the actual behavior.  For a simple function this might just be the return result but more often than not, it requires monitoring the output of the program and making a complex decision based on it.  Test developers are also responsible for creating the test harnesses and systems for their teams.

Scripters are those who have some programming ability but are not as skilled as a test developer.  They will usually know a programming language but on that is less complex than the language the product is written in.  Scripters may know Visual Basic, Perl, Javascript, or even C#.  More important than which language they know, what distinguishes them from a test developer is that their understanding of programming is less advanced.  They will understand the language syntax but their understanding of algorithms and data structures is likely less substantial. 

Scripters will often play a role where they spend a lot of time living the product.  They will use their programming skills to script the product (if it has such a feature) or to utilize tools to drive the product.  The tools could be UI toolkits like Visual Test or something written by test developers.  Because scripters have an understanding of programming they will be able to have a deep understanding of the product.  They will be able to quickly determine the root cause of a failure and see relationships between seemingly unrelated bugs.  Scripters play an important role on any team.  They often handle the bulk of the test triage efforts and closely monitor the needs of the users of the product.

In some teams the responsibility for setting up machines, installing builds, and executing test cases falls to a group I'll call Runtime Testers.  This role involves no programming.  Runtime testers usually don't have any significant programming knowledge and if they do, they don't often get a chance to utilize it.  They play a fundamental role, however.  Because they are the ones running all of the tests, they often have the best knowledge of the state of the product.  If you want to know what really does and does not work, ask one an execution specialist. 

Sometimes this role is relegated to merely clicking buttons.  They run an automated test and log some errors.  This is a waste.  They should be tasked with understanding the product and playing with it.  As I've stated many times in the past, there is a critical need for manual testing on most project.  These are the people best equipped to do that.  To get maximum value out of runtime testers, they should be encouraged to live the product.  To become experts in the functionality the way a customer might use it.  In doing so they can become advocates for the customer and find the usability issues and corner cases that automation cannnot easily find.

Each of these three roles is critical on a test team.  If you build a team with only one or two of the roles, you'll be missing something fundamental.  Each project will require a different mix of people.  Some need more test developers, others more runtime testers.  The importance of these roles comes into play not only when you task your team but also when you hire.  If someone is to be a runtime tester, hiring a CS grad may not be the best idea.  They will likely be bored and quickly move on.  Hiring a self-educated computer geek may be a better fit for the role.  Likewise, if you need test developers, hiring someone who only knows perl is probably not a good fit.  Instead, hire someone who is truly a developer.  My rule of thumb is that if they couldn't cut it as a dev, don't hire them as a test dev.