Cem Kaner: Testing is Critical in Development Process

Before I interviewed Cem Kaner, I thought a career involving software testing meant learning a lot about, well, software testing.

But take a look at the curriculum vitae of Kaner, a Professor of Software Engineering at the Florida Institute of Technology, and  you’ll also see degrees and academic work in everything from experimental psychology, math and philosophy to law and psychophysics, the latter being “the branch of psychology that deals with the relationships between physical stimuli and sensory response.”

Kaner, who is Director of Florida Tech’s Center for Software Testing Education & Research, has taken this multi-disciplinary approach because he’s obsessed with one thing: improving the satisfaction and safety of those who buy, use and develop software.

So Kaner seemed like a good person to quiz about what works in software testing and what doesn’t. What follows is the first installment of a two-part series of excerpts from a lengthy interview I did with him via email. 

What testing approach produces the best code when all is said and done? 

Kaner: I don’t think there is one best approach. I’ve seen excellent testing and excellent code delivered from each approach, and I’ve seen bad results from each. 

Testing helps managers manage their products. It helps managers understand status: For example, testers can provide information about how much is actually there, how well is it working, what course corrections are needed and how much it will cost if they are not made. 

If the testing effort isn’t providing the manager with useful information, either the testing effort is following an ineffective approach or the manager is incompetent (incapable of dealing with difficult facts). Both of these are genuine risks. They arise in real projects and have serious impact. 

What is your view of Agile-based methods for testing software? Do they work? 

I’m a huge fan of test-driven development. But we need a terminology check here. TDD, as I understand it, involves an extensive set of unit tests, written by the programmers who are writing the code, while they are writing that code. It guides the details of their implementation. It is the foundation of their refactoring. It is the safety net for future maintenance. 

Code developed this way costs a lot less in system testing. You still do have to do system testing, to discover whether the full application provides value to the people who use it and plays nicely with the other software that interacts with it. But TDD catches and stomps the simple bugs that otherwise waste enormous amounts of system-test time and destroy project schedules. 

The problem with TDD is that only about 10 percent of the groups that do “agile development” do TDD, or any level of organized unit testing … 

A lot of development methodologies are called “agile” that don’t involve extensive unit testing, but I think this is a fundamental difference. I wish we could call them something else, to make discussions clearer … 

With the right project manager, a non-TDD approach might be perfectly effective. With a different project manager, the same approach can fail. Software development is a human-intensive activity, agile development even more so than traditional approaches, and the project manager’s skill at fostering collaboration between different types of efforts in the face of shifting circumstances is critical. Different project management methodologies bring out the best (and the worst) in different project managers. 

Next week: Dr. Kaner assesses some software testing tools and talks about the role of automation.