My thoughts on QA drew one amazing comment by Justin Bozonier. It’s reprinted here for your pleasures and enjoyments! (Quotes from my original post are in italics.)
This is going to read really harsh. I’m just being as succinct as possible.
“If you’re running a web site, your users are going to find bugs faster than your testers.”
Really? At what rate are you shipping bugs? More than one person can find in 8 hours a day 5 days a week? Damn.
“They run more configurations and use cases than you ever will.”
And more than I will ever care about. 90-95% of my users are on four browsers. Why are you even releasing features for such marginal returns? Ever notice many bugs will remain in production for months (or eternity)? If that “feature” that’s broken was really necessary I’d expect the fix to have been prioritized quicker.
“If you’re using TDD, then what will testers do?”
This is an interesting point to consider. If you use TDD are you guaranteeing the system works the way the developer intended or the business owner? Your question implies it will perfectly match the intent of the business.
Also, you’ve still got the role but you’ve just tacked it onto what you expect your developers to do. Where does your pessimistic force come from?
Does 100% TDD mean important exceptional cases are necessarily covered? (Think null conditions, exception handling, slow databases, etc.).
“Manual testing doesn’t scale. Think about it logically: the quality of manual testing gets worse as a product grows.”
This makes it sound as if the quality of ONLY manual testing gets worse as a product grows. Also manual testing does scale… but it’s expensive. To your point, the longer the feedback loop the more expensive it is to have a bug. Furthermore, every run of manual testing costs MUCH MUCH more than automated testing.
The quality (false negatives/no bugs found when they do exist) of manual testing getting worse as a product grows implies that this doesn’t happen with TDD. However, I’ve personally seen that it gets harder there as well. Done improperly, your automated tests can make your system brittle and harder to refactor (and lower costs in other ways).
“Using testers lengthens your feedback loop, yet short feedback loops are critical for quality software development, so how does that make sense?”
I read this as you saying, “Using manual testers causes your short term costs to increase X%, yet automated tests only increase your costs Y%”. Until you have those numbers this is an inadequate argument.
“short feedback loops are critical for quality software development”
Are there no examples of businesses using longer feedback loops making quality software than businesses using shorter feedback loops? I guess those guys going to the moon all died.
“Using testers promotes a waterfall mentality, and doesn’t promote code ownership.”
Here you’re using a logical fallacy called “Appeal to the people”. It’s very popular to deride waterfall. But waterfall was never the problem, it was choosing waterfall when it was the inappropriate tool for the job. Given that a start up’s goal is to find a sustainable business ASAP, and that no plan survives first contact with the customer, I think it’s safe to say it doesn’t work for most start ups.
From a psychological perspective, as a developer, knowing that you’re “flying without a safety net” is a giant motivator for encouraging thoroughness and attention to detail.
There’s no safety net? I thought we were assuming all of the developers TDD all the things they can think of?
My point is that this dichotomy you’re creating enforces the mindset that these tools are mutually exclusive. It’s a false dichotomy. Someone does need to be your quality champion. They need to push all your developers to TDD everything they can think of. They need to help think of bugs from a higher level perspective. Sometimes a developer’s nose is too close to the grindstone to see the whole workshop is actually on fire. In those times, you need less grinding and more water.
To be fair, I’ve never seen this happen either. From personal experience I know TDD isn’t a panacea either. 