Thursday, March 25, 2004

Test Developers Are Real Developers

Through a few twists of fate, I ended up at Microsoft as a test developer (lead).  It’s not something I ever considered doing before landing here and I’m sure it is not something a lot of you have thought much about.  It is the goal of this post to introduce you to who test developers are, where they fit into the team, and what sort of work they do.

Most people seem to have a view of the software world divided into two sorts of people:  developers and testers.  Devs are the real heroes of software.  They get to write code.  Testers are a necessary evil.  They help to keep the developers from screwing up too badly but don’t produce any positive results.  I won’t go into the role of test in this post.  I’ll save that for another day.  The thing is, most people think of testers as the people who push a lot of buttons and try to break things.  They run tests.  Perhaps they even cobble together some script and batch files or a VisualTest “program” to help them out.  Those sort of testers do exist.  I’ll call them runtime testers or software test engineers (STEs).  At Microsoft, however, a lot of our testing ranks are filled with the rare breed known as the Test Developer (SDE/T).

So what is this test developer then?  Is he someone who just isn’t quite good enough to be a “real” developer?  I posit that the answer is no.  A good test developer is as good as any developer writing production code.  A good test developer is worth his weight in gold.  A test developer’s job is to develop software which exercises a product to ensure its correctness and gauge its stability.  He has to exercise every corner of the code before it ships and usually before it is documented.  On my team, test developers are developers who know how to test, not testers who know how to develop.

In my business, which is multimedia (audio and video rendering), we produce not polished applications but the APIs which underlie such programs as Windows XP Media Center Edition and Windows Media Player.  Countless other applications use our APIs underlying their games, video editing suites, and media players.  Our work product forms the building blocks of many an application.  Whenever you see video or hear sound from your computer, we’re probably underneath doing something.

So, who is it that makes sure that these APIs actually work as advertised?  It is the role of the test developer to make sure that developers using our APIs have a pleasant experience.  We are the front line of defense for the developers out there in the “real world” trying to use Windows to make a buck or two.  We are the first ones to try to implement anything on top of those interfaces.  It is our job to verify that they work as advertised.

We accomplish by writing a lot of code to exercise the interfaces.  While it starts with the fairly mundane testing of each method, it doesn’t end there.  It includes writing sample code (which may ship in the SDK) to do exactly what those coming after will try to do.  It involves trying to simulate the end-to-end scenarios this component is likely to be used for.  Most interesting, is trying to validate that the right action just took place.  Anyone can call an API and determine if it reports that it did the right thing.  It is a whole other ballgame to try to verify that the audio mixing component you are testing really did mix the audio correctly or that the video decoder really output the picture you expected and not some unintelligible pattern of greens and pinks.  The role of the test developer is to do all of this, and to do it without an SDK to guide him.  He’s the first one trying it, after all.

Test developers are tasked with one thing:  writing code and writing it first.  They are always breaking new ground:  going where no one has gone before.  It is a challenging and yet rewarding endeavor.  Great software is not written by some developers and their unit tests.  It takes someone to go beyond the granular and ensure that the whole package works together.  When your deliverable is not a whiz-bang UI but rather a set of building blocks for someone else, that task falls upon the shoulders of the test developer.

That is probably enough for now.  Anyway, if you are out there looking for work as a developer, don't skip over all those SDE/T jobs.  Our code may not ship in the box but it's not a job for someone who isn't good enough to be a “real” dev.


[This posting is provided "AS IS" with no warranties, and confers no rights.]

Wednesday, March 10, 2004

Hello World

   I guess everyone has to start off with a first post.  Here's mine.  Let's see how I do.  In the infamous words of Admiral James Stockdale, “Who am I?” and “Why am I here?”  My name is Steve Rowe and I'm a Development Lead in a test organization.  We work on some of the audio and video pipelining in the Windows operating system.  I expect to cover a lot of topics in this blog.  I plan to write about digital video.  Perhaps some primers about how it works, what the surfaces look like, etc.  I'll probably also rant about testing from time to time.  The role of testers in an organization is something I often think about.  Expect some words from me about how test should be viewed.  It is my opinion that testing is often seen as second-class.  Either it is an after-thought or it is a necessary evil.  Either way, testing is not given the respect within the organizatioin it deserves.  A solid test team will make a product.  A poor test team will destroy a project run even by the best developers.  I have interest in design and project management methodologies.  I plan to say a few words about design patterns, scrum, agile development, etc.  Finally, I'm currently taking a computer architecture class at the University of Illinois Urbana Champaign.  I suspect that I'll be speaking a bit about how the underlying architecture of a product may affect your code.  So, that's who I am and why I'm here.  Let's see how well I do about following through on this.