Best Software Essays of 2004

Joel On Software is compiling the top essays on the subject of software for the year 2004.  The nominations are up now at http://discuss.joelonsoftware.com/default.asp?best04.  Go give them a read.

Code Editor Learning Curves

A friend of mine put this together.  It's a graphical representation of the learning curve for some common code editors.  It seems quite accurate.

Enjoy the Silence

I just recently got done with a round of upgrades to my computer.  I didn't increase the amount of RAM or get a new processor.  No, I got new fans.  The difference is amazing.  When I built the computer I has gone with a high-quality motherboard, case, fast processor, good RAM, etc. but I'd skimped when it came to the power supply and cooling system.  The machine made a lot of noise.  I have now replaced the power supply with an Antec SmartPower 350, the stock AMD cpu fan with a CollerMaster XDream II, and the north bridge fan with a Zalman heat sink.  Total cost:  about $45.  These are not top-of-the-line parts but they are also not the cheapest you can buy.  This is now probably one of the quietest machines I've owned since the introduction of fans into the PC.  If your machine is loud, think about following my lead.  Upgrade your fans.  There is a lot of difference to be heard.

Approaches To Unit Testing

    I recently was involved in a discussion about unit testing.  I'll simplify the issues.  There are many more aguments each way and the area is more complex than presented here.  I will lay out the supposed advantages and disadvantages of side of the issue.  
    Unit tesing has been made quite popular lately with the advent of XP (eXtreme Programming).  The idea is fairly simple.  Have developers write tests to verify that their code works as intended.  It is after this point, that views diverge.  Well, some diverge before that and think that developers shouldn't be bothered to write tests but that, as they say, is a topic for another day.  The XP community seems to think that unit testing should be done at a very granular level.  Each individual function or object should be tested.  Others think that the unit tests should be more wholistic.  
    Granular unit testing is often synonymous with unit testing.  In it, developers test their code directly, trying each input and failing if the expected output isn't returned.  Often scaffolding is used to instantiate the object or call the function apart from the surrounding code.  Mock objects might also be used to insulate the real code from its reliance upon systems below, above, or beside it.  The advantage of this technique is that it is fast and thorough.  Each function, method, or class can be tested fully.  Each piece of code, even those difficult or impossible to reach from the public interfaces, can be tested.  The disadvantage is that testing each piece in isolation doesn't test the system as a whole.  It doesn't test the interaction between the code that will be interacting in the real world.  It is also not useful to test the shipping binary.  As each object is tested by standalone code, you don't get to see how the whole system really works.
    The other form of unit testing is testing only from exposed interfaces.  This is sometimes known as functional testing.  The only item being tested is the shipping binary and the only entry points used are those available on that shipping binary.  The advantages of this type of testing are that it tests the whole system.  It tests the interaction between each part of the system as it will be used in the real world.  These tests are also easily utilized by the test team to run as part of their testing suite.  The disadvantage is that it can be hard or even impossible to test many of the system internals.  Sometimes a simple interface can have a lot of code hidden behind it.  It also requires a greater amount of code to be written before the testing can even begin.
    In this particular discussion that I had, the advocacy was for the second type of testing only.  I advocated a blended approach thinking that would cover all bases.  It was argued that the granular form of testing tests at too low a level--that most bugs are found in the interaction between components and not in the functions themselves.  It therefore made little sense to even write them.    
    What do you think?  Do isolated unit tests have much return on investment or are they better off left undone?  Have you had any experience with either of these approaches?  If so, did it work out well or poorly?

MPEG-4 Is Dead

 Just a quick link for today.  An article on video formats I found interesting.  I'll have something on XP and Test developers soon.  Watch this space. 

MPEG-4 Is Dead.  Or so says Jan Ozer of StreamingMedia.com. 

http://www.streamingmedia.com/article.asp?id=8605:

Before MPEG-4 was introduced, three streaming technologies enjoyed widespread use: RealVideo, Microsoft's Windows Media Video, and Sorenson Video. MPEG-4 offers lower quality than any of them.

In addition to offering lower quality, MPEG-4 also sports the obligation to pay royalties, not only on encoders and decoders, but also on content. Given that decoders for the other three have always been free, it's doubtful that this costs more, looks worse "value proposition" will win many takers in the streaming media space.
...
Finally, it's hard to minimize the importance of the DVD Forum's provisional approval for Microsoft's VC-9 technology, essentially Windows Media Video 9, along with two other technologies, H.264 and MPEG-2, as mandatory on next-generation playback devices.

This announcement has three significant implications. First, it forces the committee-driven standards to compete primarily on performance, something neither has had to do yet. Second, it opens the door for proprietary standards to compete against committee standards in other markets, whether digital television or Dick Tracy-like video watches. The combined impact of these realities will either force committee members to work harder and faster or spell the death of committee-based video codec standards.

 

Test Developers Are Real Developers

Through a few twists of fate, I ended up at Microsoft as a test developer (lead).  It’s not something I ever considered doing before landing here and I’m sure it is not something a lot of you have thought much about.  It is the goal of this post to introduce you to who test developers are, where they fit into the team, and what sort of work they do.

Most people seem to have a view of the software world divided into two sorts of people:  developers and testers.  Devs are the real heroes of software.  They get to write code.  Testers are a necessary evil.  They help to keep the developers from screwing up too badly but don’t produce any positive results.  I won’t go into the role of test in this post.  I’ll save that for another day.  The thing is, most people think of testers as the people who push a lot of buttons and try to break things.  They run tests.  Perhaps they even cobble together some script and batch files or a VisualTest “program” to help them out.  Those sort of testers do exist.  I’ll call them runtime testers or software test engineers (STEs).  At Microsoft, however, a lot of our testing ranks are filled with the rare breed known as the Test Developer (SDE/T).

So what is this test developer then?  Is he someone who just isn’t quite good enough to be a “real” developer?  I posit that the answer is no.  A good test developer is as good as any developer writing production code.  A good test developer is worth his weight in gold.  A test developer’s job is to develop software which exercises a product to ensure its correctness and gauge its stability.  He has to exercise every corner of the code before it ships and usually before it is documented.  On my team, test developers are developers who know how to test, not testers who know how to develop.

In my business, which is multimedia (audio and video rendering), we produce not polished applications but the APIs which underlie such programs as Windows XP Media Center Edition and Windows Media Player.  Countless other applications use our APIs underlying their games, video editing suites, and media players.  Our work product forms the building blocks of many an application.  Whenever you see video or hear sound from your computer, we’re probably underneath doing something.

So, who is it that makes sure that these APIs actually work as advertised?  It is the role of the test developer to make sure that developers using our APIs have a pleasant experience.  We are the front line of defense for the developers out there in the “real world” trying to use Windows to make a buck or two.  We are the first ones to try to implement anything on top of those interfaces.  It is our job to verify that they work as advertised.

We accomplish by writing a lot of code to exercise the interfaces.  While it starts with the fairly mundane testing of each method, it doesn’t end there.  It includes writing sample code (which may ship in the SDK) to do exactly what those coming after will try to do.  It involves trying to simulate the end-to-end scenarios this component is likely to be used for.  Most interesting, is trying to validate that the right action just took place.  Anyone can call an API and determine if it reports that it did the right thing.  It is a whole other ballgame to try to verify that the audio mixing component you are testing really did mix the audio correctly or that the video decoder really output the picture you expected and not some unintelligible pattern of greens and pinks.  The role of the test developer is to do all of this, and to do it without an SDK to guide him.  He’s the first one trying it, after all.

Test developers are tasked with one thing:  writing code and writing it first.  They are always breaking new ground:  going where no one has gone before.  It is a challenging and yet rewarding endeavor.  Great software is not written by some developers and their unit tests.  It takes someone to go beyond the granular and ensure that the whole package works together.  When your deliverable is not a whiz-bang UI but rather a set of building blocks for someone else, that task falls upon the shoulders of the test developer.

That is probably enough for now.  Anyway, if you are out there looking for work as a developer, don't skip over all those SDE/T jobs.  Our code may not ship in the box but it's not a job for someone who isn't good enough to be a “real” dev.

 

[This posting is provided "AS IS" with no warranties, and confers no rights.]

Hello World

   I guess everyone has to start off with a first post.  Here's mine.  Let's see how I do.  In the infamous words of Admiral James Stockdale, “Who am I?” and “Why am I here?”  My name is Steve Rowe and I'm a Development Lead in a test organization.  We work on some of the audio and video pipelining in the Windows operating system.  I expect to cover a lot of topics in this blog.  I plan to write about digital video.  Perhaps some primers about how it works, what the surfaces look like, etc.  I'll probably also rant about testing from time to time.  The role of testers in an organization is something I often think about.  Expect some words from me about how test should be viewed.  It is my opinion that testing is often seen as second-class.  Either it is an after-thought or it is a necessary evil.  Either way, testing is not given the respect within the organizatioin it deserves.  A solid test team will make a product.  A poor test team will destroy a project run even by the best developers.  I have interest in design and project management methodologies.  I plan to say a few words about design patterns, scrum, agile development, etc.  Finally, I'm currently taking a computer architecture class at the University of Illinois Urbana Champaign.  I suspect that I'll be speaking a bit about how the underlying architecture of a product may affect your code.  So, that's who I am and why I'm here.  Let's see how well I do about following through on this.