Three Reasons To Consider Being a Test Developer

            When it comes to careers in the world of software most people think of programmers or what are more formally known as developers.  Developers are the people who write the software which is consequently sold or utilized by the organization.  I’ll call them dev-developers to distinguish them from test developers.  Sometimes people will also think of testers.  Testers are the people who don’t program but run the software written by developers to find bugs.  What gets lost in the shuffle is a specialized class of developers who are also testers.  We like to call them test-developers.  These are the people that write software to test software.  It is they who will be the subject of this essay. 

            The term “test developer” is sometimes used to refer to a tester who knows a scripting language like Javascript or Perl or maybe even knows VB.  Usually this person has no formal training and takes on only simple tasks.  That is not what I refer to in this essay.  The test developers I am referring to are a specialization of developers.  They write complex code in production languages utilizing computer science techniques.  See my previous essay on why test developers are real developers. 

            So, why should you consider becoming a test developer?  Why not just become a dev-dev instead of a test-dev?  That is the subject of this essay.  There are three primary reasons to become a test-dev rather than a dev-dev.  These are that it makes you a better programmer, the code you write is more broad, and it is at a sweet spot in the development cycle.

            Programmers who are or have been test developers are, on average, better programmers than those who have not.  They have a feel for what is likely to go wrong with software and so code for failure instead of coding for success.  All too often those who have not been testers write code until it works and then stop.  They write so that it can work but not so it will always work.  Test-developers have practice breaking programs and so know where they will encounter problems.  They are thus more successful at anticipating what error conditions may happen and writing resilient code.

            Secondly, test developers tend to write code which exercises the product at a higher level.  Instead of focusing all of your effort on a way to make a widget spin, instead you get to see what happens if multiple widgets spin at the same time or how spinning widgets interact with the IFoo class.  Test developers write code that exercises the product as a whole which is often more interesting and more rewarding than spending time optimizing one little corner case.  You get a more wholistic view of the product which leads to better understanding of how the various pieces interact.  Additionally, you often get to do more fun work like determining ways to break the system, put various pieces together, analyze its performance, etc.

            Finally, test development is at a sweet spot in the product release cycle.  Dev-developers work furiously in the early cycle getting their pieces code complete.  They then work furiously late in the cycle fixing the final few bugs.  The end result is often very long hours.  Test developers, on the other hand, are generally under less pressure.  In the early product cycle, you can only work as fast as code becomes available to you.  In the late cycle, your tests are already in place.  If they aren’t, it is too late to add them.  I don’t mean to imply that test developers don’t work hard.  They do.  They just tend to feel less deadline pressure than dev-devs.

            If you have experiences as a test developer that you feel may benefit someone trying to make a career decision involving test development, please share them in the comments section.

So You Want to Be a Test Developer

            So you have an interest in becoming a test developer, eh?  You’ve come to the right place.  In this paper I will lay out what I think is the best path to take to becoming a test developer or really any sort of developer.  Test developers are really just developers who happen to report in the test org. Becoming a good test developer requires learning not only a programming language but also what to do with it.  You’ll need to learn not only C++ but also about operating systems, algorithms, object-oriented programming, etc.  Becoming a test dev won’t be easy.  It is going to take a lot of reading and a lot of practice.  If you stick with the regimen, you’ll be a real developer.  You’ll have much of what someone would have if they had acquired a CS degree.  If not, well, you’ll be where you are today. 

Starting With a Language

            The place to start is obviously with a language.  If you don’t know how to program, you won’t be much of a programmer.  It is important to know, however, that knowing a language is just the beginning.  Too many people equate knowing C/C++ with knowing how to program.  Sadly, this is not the case.  Think of an author as an example.  Learning the English vocabulary and the rules of grammar will make you a proficient English speaker.  That does not, however, make you a great writer.  Mark Twain wasn’t a great writer because he knew how to diagram sentences.  Knowing what to say is even more important than knowing how to say it.  It is the same way with a programming language.  You can know C/C++ forward and backward and still be a bad programmer.

            The first decision you should make is what language to use to start your journey.  As you progress, you will probably pick up other languages but you have to start somewhere.  You should start with a language that encourages you to do the right things.  C# is an easy language to begin with.  C is simple and easy to learn.  C++ is its more complex big brother.  Most system-level work is done in C++.  That’s probably where you want to start.  If something like VB or Perl strike your fancy, you can use them but know that you’ll be limited in where you can utilize your skills.

            To learn C++, I recommend starting with a book.  The two best ones I know of are C++ Primer by Stanley Lippman and Thinking in C++ by Bruce Eckels.  I prefer these over the “24 Hours” or “21 Days” sort of books.  They may claim to teach you C++ quickly but they won’t.  C++ is huge.  It will take some time to master.  Pick one and read it cover to cover.  Do the exercises if you are that sort of person.  These are both thick books.  Don’t be intimidated.  They’ll both cover everything you need to move onto the next phase of programming.

Don’t stop there though.  You need to put this into action for it to make sense.  Come up with a project and start implementing it.  The project can be anything.  Implement a tool you’ve been itching to have, write a game, whatever.  I started by trying to implement the game of Robo Rally.  I never finished.  You might not either.  The point isn’t to complete the project.  It is to come to an understanding of how the parts work together.  Books teach us to program in discrete little packets.  Each packet makes sense atomically.  The point of writing a real program is to understand how these packets interact.

If you want to become wiser in the ways of C++, there are several other books you might want to read including Effective C++ by Scott Meyers and Inside the C++ Object Model by Stanley Lippman.  Neither book is essential but both will help make you a better programmer.

Going Beyond Language

            As I said, learning a language is just a first step.  Once you are here, you are proficient in some language (C++ most likely).  You are conversant in a language.  You are still a terrible writer.  The next step is to learn what makes a good program. 

This is where things get more complicated.  Learning pointers is hard but understanding algorithms is harder.  If your math is weak, you’ll struggle with some of these concepts.  Fear not.  None of this is rocket science.  It just may take you a bit longer.  If necessary, put down the book you are reading, read some math, then come back.  This should help you get further.  If you run into a roadblock, learn more math and continue until you get through.

There are three major subjects you need to come to understand.  The first is data structures and algorithms.  The second is operating systems and processors.  Finally, there is object-oriented programming.  Once you have these three areas understood, you’ll be well on your way to becoming a good programmer.

Algorithms are what make your programs run fast or slow.  When you are dealing with only a small amount of data, the algorithm you use doesn’t matter too much.  However, when you are dealing with hundreds of thousands or millions of items, the algorithm can matter a lot.  It is important to understand the types of algorithms available and to understand how to measure their efficiency.  Different ways of sorting would be an example of algorithms.  Data structures are ways to organize large amounts of data and maintain accessibility.  Examples are linked lists, trees, hash tables, etc.  Two books which are used a lot for this are Introduction to Algorithms by Thomas Cormen and Algorithms in C++ by Robert Sedgewick.

Understanding how processors and operating systems work will help you understand why the language and the OS do what they do.  Many things that C/C++ do make more sense once you understand how the underlying system works.  Understanding the OS and being able to read assembly language will come in invaluable when it comes time to debug.  The best OS book I know of is Operating System Concepts by Abraham Silberschatz.  It is accessible and covers the critical topics.  For processor/assembly topics, look to Computer Organization and Design by David Patterson. 

Finally, there is object-oriented programming.  Learning about OO techniques will help you understand how to write a program that is maintainable and extensible.  The best place to start is with Design Patterns by Gamma et al.  I suggest using to book to understand how patterns work in general, not to memorize every pattern.  A good book showing the application of patterns and good OO design techniques is Design Patterns Explained by Alan Shalloway.

Where to Next? 

            So you read all 5,000 pages and you still want more?  What can you study next?  The tree branches from here.  There are lots of possibilities.  Some good places to look are for general programming are debugging and good programming practices.  You can also start going down a specific path such as windows programming, graphics, ATL/COM programming, etc.

            Programmers spend an inordinate amount of time debugging the code they just wrote or even code written by others.  Debugging is an art, not a science.  The only way to get good at it is practice.  There are some books you can read which will give you a jump start though.  The best of these is Debugging Applications by John Robbins.

            Learning how to become a better programmer also takes practice but there are a few books written which give you some good pointers.  I won’t list them all here.  Everyone raves about Code Complete by Steve McConnell.  I don’t.  I find it long-winded.  A better book is The Practice of Programming by Brian Kernighan and Rob Pike.  Another book which is really useful is Refactoring by Martin Fowler.  He explains how to take old code and make it better.  Like Design Patterns, I find the first several chapters more useful than the reference that makes up the bulk of the book.  Finally, go find a copy of your groups coding standards document.  Often times these are chock full of good techniques.  They will usually tell you what sort of things to avoid and what good practices to emulate are.

Formal Training

            If you decide that self-driven book learning isn’t for you, are there alternatives?  Yes.  I prefer the classic CS education over professional certificate programs, workshops, or even IT degrees.  To that end, I have found a few programs I believe are worthwhile.  Each of these will allow you to get a full degree or just take some classes.  All are based online.

 

Regis University – Offers a Bachelor of Science in Computer Science.  http://www.regis.edu/regis.asp?sctn=onl

 

Colorado State University – Offers a Master of Computer Science program.  http://www.cs.colostate.edu/cstop/csacademics/csdegrees/onlinemcs.html

 

University of Illinois at Urbana-Champaign – Offers a Master of Computer Science degree.  Students participate in on-campus classes via newsgroups and online videos.  http://online.engr.uiuc.edu/

Bibliography:

Cormen, Thomas.  Introduction to Algorithms.  Thorough introduction to algorithms.  The content is often advanced.

Eckels, Bruce.  Thinking in C++.  Focuses a lot on STL which isn’t used much at Microsoft.  You can download a free copy here:  http://mindview.net/Books/DownloadSites.

Fowler, Martin.  Refactoring.  How to spot good and bad code and what to do about the bad stuff.

Gamma, Erich, Richard Helm, Ralph Johnson, John Vlissides.  Design Patterns.  The first few chapters are the most useful.  They will help you understand how OO designs should work.

Kernighan, Brian, Rob Pike.  The Practice of Programming.  Tips and techniques for being a good programmer from one of the authors of the C language.

Lippman, Stanley.  C++ Primer.  Great introduction to C/C++.

Lippman, Stanley.  Inside the C++ Object Model.  The how and why of C++.

Meyers, Scott.  Effective C++.  50 tips for writing C++.  Each tip is explained.  Along the way, you learn a lot of C++.  Skip More Effective C++ though.

Patterson, David.  Computer Organization and Design.  Covers the MIPS processor not the Intel but the concepts are the same.  Don’t get bogged down in the measurements chapter.

Robbins, John.  Debugging Applications for .Net and Windows.  Great book explaining how to debug programs on windows.  Focuses on the Microsoft programming environment but the general techniques are applicable everywhere.

Sedgewick, Robert.  Algorithms in C++.  Sedgewick has a series of Algorithms books covering everything from C to C++ to Java to Pascal.  The latest versions come in two volumes.

Shalloway, Alan.  Design Patterns Explained.  How to use design patterns in design.

Silberschatz, Abraham.  Operating System Concepts.  Good introduction to how operating systems work.

 

If you have suggestions for additional books, subjects, etc.  Please post them in the comments below.

Advice for CS Students

Joel Spolsky (of JoelOnSoftware fame) is well-spoken and often has a lot of good advice for programmers.  This recent essay gives advice to college students on what to study.  His advice consists of:

1. Learn how to write before graduating.
2. Learn C before graduating.
3. Learn microeconomics before graduating.
4. Don't blow off non-CS classes just because they're boring.
5. Take programming-intensive courses.
6. Stop worrying about all the jobs going to India.
7. No matter what you do, get a good summer internship.

If you want to find out his reasons, read the whole article.

Best Software Essays of 2004

Joel On Software is compiling the top essays on the subject of software for the year 2004.  The nominations are up now at http://discuss.joelonsoftware.com/default.asp?best04.  Go give them a read.

Code Editor Learning Curves

A friend of mine put this together.  It's a graphical representation of the learning curve for some common code editors.  It seems quite accurate.

Enjoy the Silence

I just recently got done with a round of upgrades to my computer.  I didn't increase the amount of RAM or get a new processor.  No, I got new fans.  The difference is amazing.  When I built the computer I has gone with a high-quality motherboard, case, fast processor, good RAM, etc. but I'd skimped when it came to the power supply and cooling system.  The machine made a lot of noise.  I have now replaced the power supply with an Antec SmartPower 350, the stock AMD cpu fan with a CollerMaster XDream II, and the north bridge fan with a Zalman heat sink.  Total cost:  about $45.  These are not top-of-the-line parts but they are also not the cheapest you can buy.  This is now probably one of the quietest machines I've owned since the introduction of fans into the PC.  If your machine is loud, think about following my lead.  Upgrade your fans.  There is a lot of difference to be heard.

Approaches To Unit Testing

    I recently was involved in a discussion about unit testing.  I'll simplify the issues.  There are many more aguments each way and the area is more complex than presented here.  I will lay out the supposed advantages and disadvantages of side of the issue.  
    Unit tesing has been made quite popular lately with the advent of XP (eXtreme Programming).  The idea is fairly simple.  Have developers write tests to verify that their code works as intended.  It is after this point, that views diverge.  Well, some diverge before that and think that developers shouldn't be bothered to write tests but that, as they say, is a topic for another day.  The XP community seems to think that unit testing should be done at a very granular level.  Each individual function or object should be tested.  Others think that the unit tests should be more wholistic.  
    Granular unit testing is often synonymous with unit testing.  In it, developers test their code directly, trying each input and failing if the expected output isn't returned.  Often scaffolding is used to instantiate the object or call the function apart from the surrounding code.  Mock objects might also be used to insulate the real code from its reliance upon systems below, above, or beside it.  The advantage of this technique is that it is fast and thorough.  Each function, method, or class can be tested fully.  Each piece of code, even those difficult or impossible to reach from the public interfaces, can be tested.  The disadvantage is that testing each piece in isolation doesn't test the system as a whole.  It doesn't test the interaction between the code that will be interacting in the real world.  It is also not useful to test the shipping binary.  As each object is tested by standalone code, you don't get to see how the whole system really works.
    The other form of unit testing is testing only from exposed interfaces.  This is sometimes known as functional testing.  The only item being tested is the shipping binary and the only entry points used are those available on that shipping binary.  The advantages of this type of testing are that it tests the whole system.  It tests the interaction between each part of the system as it will be used in the real world.  These tests are also easily utilized by the test team to run as part of their testing suite.  The disadvantage is that it can be hard or even impossible to test many of the system internals.  Sometimes a simple interface can have a lot of code hidden behind it.  It also requires a greater amount of code to be written before the testing can even begin.
    In this particular discussion that I had, the advocacy was for the second type of testing only.  I advocated a blended approach thinking that would cover all bases.  It was argued that the granular form of testing tests at too low a level--that most bugs are found in the interaction between components and not in the functions themselves.  It therefore made little sense to even write them.    
    What do you think?  Do isolated unit tests have much return on investment or are they better off left undone?  Have you had any experience with either of these approaches?  If so, did it work out well or poorly?