Wednesday, January 19, 2005

Three Reasons To Consider Being a Test Developer

            When it comes to careers in the world of software most people think of programmers or what are more formally known as developers.  Developers are the people who write the software which is consequently sold or utilized by the organization.  I’ll call them dev-developers to distinguish them from test developers.  Sometimes people will also think of testers.  Testers are the people who don’t program but run the software written by developers to find bugs.  What gets lost in the shuffle is a specialized class of developers who are also testers.  We like to call them test-developers.  These are the people that write software to test software.  It is they who will be the subject of this essay. 

            The term “test developer” is sometimes used to refer to a tester who knows a scripting language like Javascript or Perl or maybe even knows VB.  Usually this person has no formal training and takes on only simple tasks.  That is not what I refer to in this essay.  The test developers I am referring to are a specialization of developers.  They write complex code in production languages utilizing computer science techniques.  See my previous essay on why test developers are real developers. 

            So, why should you consider becoming a test developer?  Why not just become a dev-dev instead of a test-dev?  That is the subject of this essay.  There are three primary reasons to become a test-dev rather than a dev-dev.  These are that it makes you a better programmer, the code you write is more broad, and it is at a sweet spot in the development cycle.

            Programmers who are or have been test developers are, on average, better programmers than those who have not.  They have a feel for what is likely to go wrong with software and so code for failure instead of coding for success.  All too often those who have not been testers write code until it works and then stop.  They write so that it can work but not so it will always work.  Test-developers have practice breaking programs and so know where they will encounter problems.  They are thus more successful at anticipating what error conditions may happen and writing resilient code.

            Secondly, test developers tend to write code which exercises the product at a higher level.  Instead of focusing all of your effort on a way to make a widget spin, instead you get to see what happens if multiple widgets spin at the same time or how spinning widgets interact with the IFoo class.  Test developers write code that exercises the product as a whole which is often more interesting and more rewarding than spending time optimizing one little corner case.  You get a more wholistic view of the product which leads to better understanding of how the various pieces interact.  Additionally, you often get to do more fun work like determining ways to break the system, put various pieces together, analyze its performance, etc.

            Finally, test development is at a sweet spot in the product release cycle.  Dev-developers work furiously in the early cycle getting their pieces code complete.  They then work furiously late in the cycle fixing the final few bugs.  The end result is often very long hours.  Test developers, on the other hand, are generally under less pressure.  In the early product cycle, you can only work as fast as code becomes available to you.  In the late cycle, your tests are already in place.  If they aren’t, it is too late to add them.  I don’t mean to imply that test developers don’t work hard.  They do.  They just tend to feel less deadline pressure than dev-devs.

            If you have experiences as a test developer that you feel may benefit someone trying to make a career decision involving test development, please share them in the comments section.

27 comments:

  1. Great post, Steve. One question: In an earlier post you said that C++ was the language of choice for you in your role as an SDE/T. Is that specifically for your group, or are there other SDE/T positions within MS that would primarily use, say, C#?

    ReplyDelete
  2. For my group the language of choice is C++. That is probably true of most test-development positions today at Microsoft but certainly not all of them. The use of C# as a development platform is increasing and with it comes the need for test-devs that work primarily in C#. I see the use of C# by test-developers increasing in the future as the Windows platform becomes more C#-oriented with technologies like Avalon and WinFX.

    ReplyDelete
  3. Steve - Since all, or a majority portion, of Microsoft products are dependent upon the inherent advantages of binary speed versus the .NET JIT Model, can an argument even be made for the total separation from binary technologies in Microsoft products such as COM?


    As an aspiring Microsoft Employee, I have been cracking the books on C++/COM/ATL/WTL, etc. I see great advantages in speed over C#. (GDI+ as an example versus standard WIn32 API GDI drawing). Am I on the right track of preparation for an SDET or SDE position at Microsoft?


    Being an SDET/SDE for Microsoft is within my reach. I once broke the security of an IBM S/390 Mainframe running MVS/ESA over a Hydra Multiplexer while in my CICS class in college. I LOVE TO BREAK THINGS! (and fix them too...) Any openings for SDET's on your team? (Gretchen Ledgard gledgard@microsoft.com knows who I am.)


    Cheers.

    ReplyDelete
  4. http://seattletimes.nwsource.com/html/businesstechnology/2002155249_mslayoffs20.html


    Microsoft lays off 62 testers


    By Brier Dudley


    Seattle Times technology reporter



    Microsoft is laying off 62 test engineers in the second round of cuts hitting Windows testers in the past five months.


    ReplyDelete
  5. I started writing a response to the C#/C++ question and it grew long. I'll make it a full blown post.

    As for what to focus on, I don't think it matters a lot. If you know the fundamentals, you can pick up any language in a short period of time. Right now, you'll be able to sell yourself more broadly with C++/COM/ATL than with C#. More important than that is an understanding of fundamental CS principles.

    ReplyDelete
  6. I'm not familiar with the specific situation surrounding the 62 testers let go. When you see the word "test engineers" that probably refers to STEs. It is true that Microsoft is shifting its focus from STEs to SDE/Ts. See http://blogs.msdn.com/chappell/archive/2004/10/06/239001.aspx for a discussion of the differences.

    ReplyDelete
  7. Steve - Thanks. Yes, I have an excellent understanding of the fundamentals. C++/COM/ATL seems to be the core languages for core products.


    Since I am an apsiring SDE/SDET, I want to make sure I am on focus.

    ReplyDelete
  8. I've been working as a contract STE at MS on and off for a bit over two years total now, and I've come to the realization that I am going to quite likely need to be able to work at an SDET level if I'm ever going to get a blue badge. One thing that a former manager of mine told me (and I've heard reiterated elsewhere) is that one should not look at testing as a step along the path to development. Really, as is emphasized here, testing is largely considered to be its own career path. Based on what I've seen (admittedly limited though) if you advance from an STE at MS, you're probably more likely to end up a PM at some point along the line than you are to end up a dev-developer.

    ReplyDelete
  9. Steve -

    I align on the same side of the fence with you when you say - "If you know your fundamentals, you can pick up any language in a short period of time"


    After being a Micosoft technologies consultant for most of my 12 year career, I have been working with Java and J2EE for about 3 years now and can easily make the transition to C# and .NET. Also there are other common aspects like design patterns, OOP, database design, etc. Does Microsoft consider "OUT OF THE BOX" (excuse the pun)experience?


    Sent Gretchen an email yesterday about this too. Hopefully she can open a discussion topic?


    Krish

    ReplyDelete
  10. One reason not to be an SDET: you could end up working for Steve!


    :)

    ReplyDelete
  11. Thanks Greg. Love you too. :)


    Krish - I can't speak for all of Microsoft. We all hire differently. In my little corner of the world we do.

    ReplyDelete
  12. Steve

    That's encouraging. Beginning to wonder if I should look at Test Development as a career option.

    Krish

    ReplyDelete
  13. On the other hand, SDET is a kind of purgatory at Microsoft. I was one for 3 years before becoming a developer and the difference is significant.


    The career development path is clearer as a dev-dev; oftentimes, SDETs have to meet criteria both within test and dev ladders to advance - and teams sometimes abuse this to keep people from advancing.


    There is also an unfair prejudice against SDETs - during my informational interviews, I ran into this many times. "Oh, so you're a tester and now you want to write code?" Steve is right that SDETs are just as good as dev-devs (in some cases better) but until this company and others takes the role seriously all over the place, stick to being a dev-dev if you can.

    ReplyDelete
  14. What Contrarian says is sadly often true. In the past, it depended which group you were with whether they considered test-devs equal with developers or not. In my group I've worked to ensure that test-devs are respected.


    Microsoft does seem to be changing though. As has been noted elsewhere, there are changes afoot to bring more clarity to the test development position. With this comes greater potential for advancement and greater respect.


    Contrarian, other than the question, do you think being a test-dev actually hurt you in the interview process?

    ReplyDelete
  15. If you're asking whether I felt unprepared for interview questions, no, I did fine (my success rate was comparable to the norm according to a recruiter - granted this is a small sample size and I chose carefully). I think I had an edge when it came to "so how would you test this API you just wrote?"


    Where it hurt was in the informational stage; it seemed like the odds of getting a reply from a hiring manager was lower for SDE vs. SDET positions at the same level (I interviewed for both, knowing this was going to be a roadblock). But once past the bias, it's a level playing field.


    Ironically, the team I am on now really does value its SDETs; they are a part of the design process from day one and their input is as important as any PM or dev.


    I have read of plans to clarify the career ladder levels for SDETs, this would be a welcome thing for many of my former colleagues and friends who are SDETs.


    One team I used to work on did this weird thing where they changed the SDET titles to SDE in the address book if you had been there a while - to me, this actually devalued the SDET role because the implicit message is "okay, you've proven yourself to us now you can think of yourself as a developer."

    ReplyDelete
  16. The indigo team has some job openings. If you are passionate about working at Microsoft, passionate about...

    ReplyDelete
  17. What is the difference between a software test engineer (STE) and a software design engineers in test...

    ReplyDelete
  18. PingBack from http://puggy.symonds.net/~charisma/wp/?p=9

    ReplyDelete
  19. I'm going to be doing a series not on testing but on the people that carry it out. This will be a post

    ReplyDelete
  20. Are you a good enough developer to work on the Test team and interested in working for the Embedded Windows

    ReplyDelete
  21. Are you a good enough developer to work on the Test team and interested in working for the Embedded Windows

    ReplyDelete
  22. On his blog, Microsoft employee Steve Rowe discusses the top three reasons to consider being a Test Developer

    ReplyDelete
  23. PingBack from http://copyrightrenewalsblog.info/vipul-modis-blog/

    ReplyDelete
  24. On his blog, Microsoft employee Steve Rowe discusses the top three reasons to consider being a Test Developer (aka a Software Design Engineer in Test or SDET.) Reading his post prompted me to look at other entries on his blog, and I noticed he’s also

    ReplyDelete
  25. On his blog, Microsoft employee Steve Rowe discusses the top three reasons to consider being a Test Developer (aka a Software Design Engineer in Test or SDET.) Reading his post prompted me to look at other entries on his blog, and I noticed he’s also

    ReplyDelete