Here is a comment I received via e-mail in reference to my post on the Dangers of Test Automation. I found it insightful. The point is the oversights are easy and automation won't catch them. Despite a few claims that this is just a deficiency in the test plan, assuming that a test plan is perfect is a dangerous assumption. There are always going to be holes, just like there will always be bugs in the code we write. Having manual exploration gives us insurance that we will cover these holes. Once again, if all of your testing is automated, you are done finding bugs after your first run.
Here is the comment:
I was reading your post:
... and I was immediately reminded of a scene from Jurassic Park (the book, not the movie, though that was good too)
The park administrator is showing off the high-tech computer systems. He says "you can search for dinosaurs... I'll just search for all of them... and the big board will show you their current locations." Sure enough, there they all are.
Later it is revealed that, contrary to design, the dinosaurs are breeding. Someone asks why the extra dinosaurs didn't show up on the computer search.
Nedry answers (I'm paraphrasing) "The search allows you to enter an 'expected number' of dinosaurs, which helps it to run faster. It's a feature."
Premature optimization strikes again...
Of course a more logical way to write this part of the book is that each dinosaur was implanted with a tracker, but born dinosaurs obviously have no tracker.
ReplyDeletePlus if the search was limited by a value, it would have found a random (or by proximity) choice of dinosaurs. Of course tracking them without some type of ID/tracking is just silly science.
> each dinosaur was implanted with a tracker
ReplyDeleteIIRC the tracking was done by some kind of hand-wavy implementation involving motion sensors. But there was evidence that the computer was aware of the "new" children.
There was also a histogram that showed counts of dinosaurs by height. The biological engineer who "made" the dinosaurs said they were released in three separate birthing cycles. The mathematician pointed out that the shape of the "height" curve was a normal curve, which implied a breeding population... he said with a three-stage birthing cycle he would have expected a three-hump curve.
Nobody believed him.
But on a second reading I added up the counts for each data point on the histogram, and they added up to more than the number of dinosaurs given to the search. It struck me as odd that the mathematician would have missed that, but perhaps the author didn't want to give it away too early.
This is the third in my series on test harnesses. In this post, I'll talk about systems that do much...
ReplyDelete